-- Logs begin at Thu 2022-07-14 14:57:42 UTC, end at Thu 2022-07-14 16:15:58 UTC. -- Jul 14 15:33:36.383828 dv-cinder-837164-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 15:33:36.392858 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 15:33:36 2022] *** Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: nodename: dv-cinder-837164-3-iscsi Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: machine: x86_64 Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: clock source: unix Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: pcre jit disabled Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: detected number of CPU cores: 4 Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: current working directory: / Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: detected binary path: /usr/bin/uwsgi-core Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: your processes number limit is 47680 Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: your memory page size is 4096 bytes Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: detected max file descriptor number: 2048 Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: lock engine: pthread robust mutexes Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: thunder lock: enabled Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 15:33:36.399467 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 14 15:33:36.439045 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: Python main interpreter initialized at 0x55acddbb69c0 Jul 14 15:33:36.439045 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: python threads support enabled Jul 14 15:33:36.439045 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: your server socket listen backlog is limited to 100 connections Jul 14 15:33:36.439045 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: your mercy for graceful operations on workers is 90 seconds Jul 14 15:33:36.439373 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 15:33:36.439472 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: *** Operational MODE: preforking *** Jul 14 15:33:36.439748 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: *** uWSGI is running in multiple interpreter mode *** Jul 14 15:33:36.439748 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: spawned uWSGI master process (pid: 92122) Jul 14 15:33:36.439951 dv-cinder-837164-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 15:33:36.441322 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: spawned uWSGI worker 1 (pid: 92133, cores: 1) Jul 14 15:33:36.441322 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: spawned uWSGI worker 2 (pid: 92134, cores: 1) Jul 14 15:33:36.441322 dv-cinder-837164-3-iscsi devstack@keystone.service[92122]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 15:33:41.904597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 15:33:41.904597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: from cryptography.utils import int_from_bytes Jul 14 15:33:41.905401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 15:33:41.905401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: from cryptography.utils import int_from_bytes Jul 14 15:33:41.905401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 15:33:41.905401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: from cryptography.utils import int_from_bytes Jul 14 15:33:41.906073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 15:33:41.906073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: from cryptography.utils import int_from_bytes Jul 14 15:33:41.955148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 15:33:41.955317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 15:33:41.955403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 15:33:41.955491 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 15:33:41.955566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] command line args: [] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 15:33:41.955637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] command line args: [] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 15:33:41.955780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 15:33:41.955867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 15:33:41.955965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ================================================================================ {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 15:33:41.956045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ================================================================================ {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 15:33:41.956223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] admin_token = **** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.956308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] admin_token = **** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.956474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] config_dir = [] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.956553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] config_dir = [] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.956664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.956742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.956845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] config_source = [] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.956923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] config_source = [] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.957025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] debug = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.957110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] debug = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.957222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.957522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.957658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.957762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.957908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] list_limit = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] list_limit = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] log_config_append = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] log_config_append = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.958934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] log_dir = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] log_dir = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] log_file = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] log_file = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.959941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.960961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.961102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.961222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.961301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.961419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.961499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.961621 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.961734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.961849 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.962015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.962110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.962264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.962342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.962540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.962627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] publish_errors = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] publish_errors = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.963927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.964045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.964146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.964271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.964399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.964508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] standard_threads = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.964592 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] standard_threads = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.964740 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.964825 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.964981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] use_journal = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] use_journal = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] use_json = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] use_json = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] use_stderr = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] use_stderr = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.965857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] use_syslog = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.966015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] use_syslog = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.966092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.966168 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 15:33:41.966340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.966424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.966579 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.966667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.966841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.966919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.967081 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.967158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.967354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.967440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.967596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.967678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.967844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.967922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.968081 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.968167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.968329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.968406 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.968577 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.968653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.968828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.968913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.969065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.969143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.969317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.969393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.969553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.969644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.969802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.969917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.970074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.970153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.970315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.970394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.970570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.970649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.970827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.970905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.971074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.971154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.971313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.971391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.971567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.971644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.971829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.971908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.972083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.972160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.972331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.972409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.972564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.972648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.972796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.972873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.973026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.973102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.973261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.973339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.973513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.973590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.973761 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.973865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.974021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.974107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.974254 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.974367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.974504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.974608 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.974753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.974857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.974992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.975101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.975243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.975324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.975479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.975600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.975718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.975795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.975945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.976030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.976175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.976303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.976409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.976499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.976641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.976718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.976872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.976953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.977108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.977190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.977357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.977444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.977590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.977669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.977868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.977953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.978071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.978145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.978304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.978396 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.978540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.978624 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.978787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.978865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.979018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.979108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.979254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.979333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.979494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.979574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.979730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.979818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.979966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.980050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.980201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.980278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.980432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.980510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.980679 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.980764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.980915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.981000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.981153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.981232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.981385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.981471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.981646 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.981731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.981906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.982006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.982140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.982218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.982377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.982455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.982615 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.982700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.983402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.983481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.983643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.983720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.983891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.984044 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.984125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.984198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.984363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.984448 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.984598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.984683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.984831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.984908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.985187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.985266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.985433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.985524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.985675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.985760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.985948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.986030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.986191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.986270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.986432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.986512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.986668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.986745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.986905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.986982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.987142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.987221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.987407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.987485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.987649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.987728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.987891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.987970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.988132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.988210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.988375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.988452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.988613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.988692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.988848 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.988925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.989085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.989169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.989314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.989394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.989545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.989622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.989775 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.989882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.990039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.990120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.990274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.990353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.990519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.990598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.990757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.990837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.991012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.991092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.991252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.991333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.991510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.991588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.991770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.991854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.992013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.992091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.992246 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] auth.external = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.992325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] auth.external = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.992478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.992555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.992776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.992886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.993011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.993091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.993243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] auth.password = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.993322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] auth.password = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.993473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] auth.token = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.993552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] auth.token = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.993714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.993792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.993975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.994065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.994225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.994304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.994459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.994537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.994707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.994798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.994954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.995038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.995184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.995264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.995417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] credential.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.995498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] credential.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.995665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.995745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.995916 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.995996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.996162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.996242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.996400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.996478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.996643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.996729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.996877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.996956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.997142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.997220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.997398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.997484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.997637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.997715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.997915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.997998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.998201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.998359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.998526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.999114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.999289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.999370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.999535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.999627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.999797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:41.999877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.000035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] federation.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.000115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] federation.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.000277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.000364 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.000526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.000605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.000774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.000852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.001009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.001096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.001256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.001327 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.001504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.001602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.001749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.001858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.002013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.002095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.002272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.002351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.002510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.002596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.002757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.002837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.002992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.003071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.003240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.003327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.003487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.003566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.003724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.003803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.003960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.004047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.004209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.004288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.004440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.004519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.004678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.004765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.004928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.005007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.005181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.005259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.005413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.005501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.005647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.005727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.005901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.005985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.006148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.006239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.006393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.006472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.006638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.006716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.006892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.006979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.007137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.007216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.007406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.007528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.007636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.007736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.007872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.007952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.008101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.008180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.008337 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.008416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.008567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.008646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.008802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.008880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.009042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.009123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.009280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.009365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.009527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.009606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.009757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.009856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.010026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.010105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.010272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.010350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.015819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.015933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.016966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.017960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018491 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.018934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019005 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019814 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.019964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020403 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.020976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021049 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021246 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021469 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.021987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022814 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.022958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.023998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] resource.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] resource.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.024963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025403 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025482 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.025957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] role.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026250 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] role.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] role.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] role.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.026857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.027024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.027107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.027273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.027353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.027522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.027602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.027770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.027853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.028017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.028097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.028264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.028343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.028499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.028580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.028730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.028808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.028978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.029058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.029225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.029305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.029470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.029549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.029743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.029917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.030164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.030244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.030403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.030482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.030656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.030744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.030908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.030986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.031161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.031239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.031397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.031525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.031638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.031717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.031898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.031978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.032138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.032229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.032374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.032454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.032605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.032683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.032837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.032923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.033069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.033148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.033325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.033404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.033574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.033661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.033849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.033970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.034068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.034164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.034296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.034414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.034515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.034610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] token.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.034752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] token.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.034859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.034992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.035106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.035229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.035334 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.035472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.035576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.035730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.035842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.035977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.036082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.036221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.036325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.036462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.036574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.036705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.036811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.036951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.037055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.037194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.037306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.037429 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.037541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.037665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.037768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.037936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.038065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.038199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.038334 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.038437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.038541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.038638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.038785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.connection = **** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.connection = **** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.039984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.040081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.040184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.040314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.040499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.040602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.040699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.040802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.040904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.041009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.041114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.041234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.041340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.041478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.041582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.041713 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.041847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.041958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.042054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.042668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.042778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.042890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.042991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.042991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.043175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.043175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.043353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.043353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.043542 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.043654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 15:33:42.043790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 15:33:42.043790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 15:33:42.075043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.075225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.075297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.075359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.075484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.075597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.090545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.090646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.119182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.119348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.119436 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.119578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.282538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.283364 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.343010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.343252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.343347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.343571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.409434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.409731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.409843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.409918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.410173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.410262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.410327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.410389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.410630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.410722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.410789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.410991 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.436204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.436439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.436581 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.436879 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.436979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.437109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.437226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.437366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.437572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.437659 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.437792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.437898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.438031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.438250 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.449335 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.449646 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.462092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.462326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.462555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.462639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.462780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.462863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.463000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.463083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.463217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.463301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.463428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.463510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.463642 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.463854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 15:33:42.491879 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.492088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 15:33:42.492429 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.492512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.492633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 15:33:42.492719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 15:33:42.493038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.493119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.493239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 15:33:42.493392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.493657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.493741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.493940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.494024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.494280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.494540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.504892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.505171 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.505440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.505566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.505725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.505867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.506140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.506419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.514074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.514710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.517373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.517653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.517957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.518042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.518241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.518338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.518523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.518607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.518784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.518867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.519051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.519142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.519306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.519387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.519569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.519650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.519837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.519919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.520100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.520180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.520367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.520630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.547767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.548180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.549336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.549617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.549788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.550093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.554930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.555209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.555365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.555491 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.555640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.555917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.562990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.563273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.563358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.563638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.568051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.568338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.568421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.568622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.568706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.568906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.568989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.569178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.569259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.569537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.569711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.569792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.570000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.570084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.570264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.589190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.589355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.589465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.589630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.594184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.594351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.597837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.598015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.598139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.598306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.603331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.603452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.607003 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.607095 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.611187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.611293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.611488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.611573 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.611783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.611867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.612079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.612171 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.612355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.612447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.612630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.612713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.632238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.632400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.632564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.632678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.632786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.632925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.633044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.633186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.640998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.641177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.642727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.642919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.643002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.643195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.646186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.646421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.646507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.646706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.646790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.646993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.647078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.647278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.647361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.647563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.660562 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.660838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.660949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.661113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.661230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.661407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.661499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.661686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.661768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.661991 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.662075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.662273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.662365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.662624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.681970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.682453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.685237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.685733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.688607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.689115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.690244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.690761 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.692262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.692539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.692794 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.692879 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.693075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.693361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.700213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.700729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.702513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.702887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.705736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.706048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.706134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.706331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.706416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.706605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.706689 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.706951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.716216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.716506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.716789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.717217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.718017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.718297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.725258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.725534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.725812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.726114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.726197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 15:33:42.726385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.726469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.726662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.726748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.726935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.727017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.727204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.727297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.727480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.727563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.727753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.727835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.728028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.728112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.728344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.728425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.728609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.728690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.728881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.728964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.729164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.729428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.729692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 15:33:42.767405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55acddbb69c0 pid: 92134 (default app) Jul 14 15:33:42.767982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55acddbb69c0 pid: 92133 (default app) Jul 14 15:33:42.770812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc39fd-0f09-433e-96a7-e5de6e852cca None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:42.771110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc39fd-0f09-433e-96a7-e5de6e852cca None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:42.771389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc39fd-0f09-433e-96a7-e5de6e852cca None None] PATH_INFO: `/v3` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:42.775614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Thu Jul 14 15:33:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:33:51.331046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c18467-d23a-4967-a927-42de176c3953 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:51.331571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c18467-d23a-4967-a927-42de176c3953 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:51.331571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c18467-d23a-4967-a927-42de176c3953 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:51.336006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:33:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:33:51.341786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17b781-cc19-4770-894d-64767c77209a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:51.342313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17b781-cc19-4770-894d-64767c77209a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:51.342405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f17b781-cc19-4770-894d-64767c77209a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:51.410912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: INFO dbcounter [None req-4f17b781-cc19-4770-894d-64767c77209a None None] Registered counter for database keystone Jul 14 15:33:51.432028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG oslo_db.sqlalchemy.engines [None req-4f17b781-cc19-4770-894d-64767c77209a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92134) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 15:33:51.439123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writer thread running {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 15:33:51.618111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG passlib.handlers.bcrypt [None req-4f17b781-cc19-4770-894d-64767c77209a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92134) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 15:33:51.618438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG passlib.handlers.bcrypt [None req-4f17b781-cc19-4770-894d-64767c77209a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92134) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 15:33:51.673590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-4f17b781-cc19-4770-894d-64767c77209a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:33:51.675243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4f17b781-cc19-4770-894d-64767c77209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:33:51.800198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:33:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 15:33:51.974310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c18467-d23a-4967-a927-42de176c3953 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:52.043917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: INFO dbcounter [None req-84c18467-d23a-4967-a927-42de176c3953 None None] Registered counter for database keystone Jul 14 15:33:52.063749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG oslo_db.sqlalchemy.engines [None req-84c18467-d23a-4967-a927-42de176c3953 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92133) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 15:33:52.070245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writer thread running {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 15:33:52.188103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c18467-d23a-4967-a927-42de176c3953 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:52.192670 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64865186-9853-424d-be9b-4606bb57cdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:52.193341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64865186-9853-424d-be9b-4606bb57cdae admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:52.193631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64865186-9853-424d-be9b-4606bb57cdae admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:52.193949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64865186-9853-424d-be9b-4606bb57cdae admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:52.202207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64865186-9853-424d-be9b-4606bb57cdae admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:52.253694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WARNING keystone.common.rbac_enforcer.enforcer [None req-64865186-9853-424d-be9b-4606bb57cdae admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 15:33:52.254013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64865186-9853-424d-be9b-4606bb57cdae admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:52.263602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Thu Jul 14 15:33:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:33:52.269735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17b781-cc19-4770-894d-64767c77209a None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:52.284854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f17b781-cc19-4770-894d-64767c77209a None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:52.289320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a249c3-d2b5-49de-8702-3433b31961ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:52.290478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a249c3-d2b5-49de-8702-3433b31961ac admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:52.290773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a249c3-d2b5-49de-8702-3433b31961ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:52.291067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a249c3-d2b5-49de-8702-3433b31961ac admin admin] PATH_INFO: `/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:52.295688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a249c3-d2b5-49de-8702-3433b31961ac admin admin] RBAC: Authorizing `identity:get_project(project_id=d8c317cfa3fe4e0aa6bdd74dbc68a586)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:52.346704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING keystone.common.rbac_enforcer.enforcer [None req-21a249c3-d2b5-49de-8702-3433b31961ac admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 15:33:52.347245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a249c3-d2b5-49de-8702-3433b31961ac admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:52.351665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Thu Jul 14 15:33:52 2022] GET /identity/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:33:54.351612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae01de-2958-49c2-85d3-71c1239a3bd4 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:54.352116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae01de-2958-49c2-85d3-71c1239a3bd4 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:54.352116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae01de-2958-49c2-85d3-71c1239a3bd4 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:54.356272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:33:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:33:54.362176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986d9cf-7ab4-4376-bc30-f68472ff1483 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:54.362601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986d9cf-7ab4-4376-bc30-f68472ff1483 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:54.362742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986d9cf-7ab4-4376-bc30-f68472ff1483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:54.381412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c986d9cf-7ab4-4376-bc30-f68472ff1483 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:33:54.382965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c986d9cf-7ab4-4376-bc30-f68472ff1483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:33:54.391852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:33:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 15:33:54.413407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae01de-2958-49c2-85d3-71c1239a3bd4 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:54.425892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae01de-2958-49c2-85d3-71c1239a3bd4 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:54.429977 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea3ddf-a0b5-4f6b-8342-d0b906938bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:54.430608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea3ddf-a0b5-4f6b-8342-d0b906938bce admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:54.430891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea3ddf-a0b5-4f6b-8342-d0b906938bce admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:54.431167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea3ddf-a0b5-4f6b-8342-d0b906938bce admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:54.438827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea3ddf-a0b5-4f6b-8342-d0b906938bce admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:54.440605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea3ddf-a0b5-4f6b-8342-d0b906938bce admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:54.449472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Thu Jul 14 15:33:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:33:54.454734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986d9cf-7ab4-4376-bc30-f68472ff1483 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:54.463595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986d9cf-7ab4-4376-bc30-f68472ff1483 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:54.468618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf97f6-7ddc-45db-a45f-bc085dc3be0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:54.469479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf97f6-7ddc-45db-a45f-bc085dc3be0a admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:54.469762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf97f6-7ddc-45db-a45f-bc085dc3be0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:54.470072 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf97f6-7ddc-45db-a45f-bc085dc3be0a admin admin] PATH_INFO: `/v3/users/d2ffbc8434d34115805dcbc2a72dc671` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:54.471456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf97f6-7ddc-45db-a45f-bc085dc3be0a admin admin] RBAC: Authorizing `identity:get_user(user_id=d2ffbc8434d34115805dcbc2a72dc671)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:54.473267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf97f6-7ddc-45db-a45f-bc085dc3be0a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:54.474860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Thu Jul 14 15:33:54 2022] GET /identity/v3/users/d2ffbc8434d34115805dcbc2a72dc671 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:33:56.530735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b790239-d91b-4f3a-81a0-c0075ece9658 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:56.531266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b790239-d91b-4f3a-81a0-c0075ece9658 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:56.531266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b790239-d91b-4f3a-81a0-c0075ece9658 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:56.532619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:33:56.537740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7cb52-1e49-4fa4-9fa6-ee201015255d None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:56.538184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7cb52-1e49-4fa4-9fa6-ee201015255d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:56.538330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7cb52-1e49-4fa4-9fa6-ee201015255d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:56.555466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-15d7cb52-1e49-4fa4-9fa6-ee201015255d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:33:56.556642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-15d7cb52-1e49-4fa4-9fa6-ee201015255d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:33:56.570877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9ff85-8741-471d-b029-8472414d46f4 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:56.571160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9ff85-8741-471d-b029-8472414d46f4 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:56.571433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9ff85-8741-471d-b029-8472414d46f4 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:56.572762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:33:56.577678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ece8e-0934-4d44-9218-66289b52d5f3 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:56.577993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ece8e-0934-4d44-9218-66289b52d5f3 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:56.578274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ece8e-0934-4d44-9218-66289b52d5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:56.618515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG passlib.handlers.bcrypt [None req-7a8ece8e-0934-4d44-9218-66289b52d5f3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92133) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 15:33:56.618936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG passlib.handlers.bcrypt [None req-7a8ece8e-0934-4d44-9218-66289b52d5f3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92133) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 15:33:56.637054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:33:56.660097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7cb52-1e49-4fa4-9fa6-ee201015255d None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:56.664413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-7a8ece8e-0934-4d44-9218-66289b52d5f3 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:33:56.666017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7a8ece8e-0934-4d44-9218-66289b52d5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:33:56.683075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:33:56.764861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7cb52-1e49-4fa4-9fa6-ee201015255d None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:56.766558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:33:56.788497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ece8e-0934-4d44-9218-66289b52d5f3 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:56.801161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf1f0a-08bd-4acb-9132-573ef9bcaec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:56.802008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf1f0a-08bd-4acb-9132-573ef9bcaec3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:56.802304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf1f0a-08bd-4acb-9132-573ef9bcaec3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:56.802607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf1f0a-08bd-4acb-9132-573ef9bcaec3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:56.814180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf1f0a-08bd-4acb-9132-573ef9bcaec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-92cf1f0a-08bd-4acb-9132-573ef9bcaec3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:33:56.825774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:33:56.827215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:33:56.828260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:33:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:33:56.833339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf1f0a-08bd-4acb-9132-573ef9bcaec3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:56.891837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ece8e-0934-4d44-9218-66289b52d5f3 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:56.894192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:33:56.913542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:33:56.930611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3f588-d6a4-409a-a33e-b37976aee0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:56.931181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf1f0a-08bd-4acb-9132-573ef9bcaec3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:56.931501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3f588-d6a4-409a-a33e-b37976aee0d8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:56.931790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3f588-d6a4-409a-a33e-b37976aee0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:56.932068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3f588-d6a4-409a-a33e-b37976aee0d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:56.964638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385b9b-97d5-49cb-83eb-df75ce643e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:56.965307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385b9b-97d5-49cb-83eb-df75ce643e24 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:56.965588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385b9b-97d5-49cb-83eb-df75ce643e24 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:56.965891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385b9b-97d5-49cb-83eb-df75ce643e24 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:56.966713 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d385b9b-97d5-49cb-83eb-df75ce643e24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:33:56.967711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d385b9b-97d5-49cb-83eb-df75ce643e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:56.969125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d385b9b-97d5-49cb-83eb-df75ce643e24 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:56.981557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:33:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:33:56.987254 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385b9b-97d5-49cb-83eb-df75ce643e24 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:57.000187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3f588-d6a4-409a-a33e-b37976aee0d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:57.001730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3f588-d6a4-409a-a33e-b37976aee0d8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:57.088558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385b9b-97d5-49cb-83eb-df75ce643e24 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:57.103437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:33:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:33:57.109003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3f588-d6a4-409a-a33e-b37976aee0d8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:57.111694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:33:57.122598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8c9fd-a859-497a-b96d-3a610b1f2387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:57.123475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8c9fd-a859-497a-b96d-3a610b1f2387 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:57.123761 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8c9fd-a859-497a-b96d-3a610b1f2387 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:57.124036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8c9fd-a859-497a-b96d-3a610b1f2387 None admin] PATH_INFO: `/v3/users/d2ffbc8434d34115805dcbc2a72dc671` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:57.125262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8c9fd-a859-497a-b96d-3a610b1f2387 None admin] RBAC: Authorizing `identity:get_user(user_id=d2ffbc8434d34115805dcbc2a72dc671)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:57.126765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8c9fd-a859-497a-b96d-3a610b1f2387 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:57.128389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:33:56 2022] GET /identity/v3/users/d2ffbc8434d34115805dcbc2a72dc671 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:33:57.133967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8c9fd-a859-497a-b96d-3a610b1f2387 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:57.148492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:33:57.207409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3f588-d6a4-409a-a33e-b37976aee0d8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:57.232584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8c9fd-a859-497a-b96d-3a610b1f2387 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:57.240930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506474-01a0-4a37-aa81-ce28f9ceda86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:57.241729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506474-01a0-4a37-aa81-ce28f9ceda86 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:57.242045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506474-01a0-4a37-aa81-ce28f9ceda86 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:57.242327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506474-01a0-4a37-aa81-ce28f9ceda86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:57.261258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb506474-01a0-4a37-aa81-ce28f9ceda86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:57.262774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb506474-01a0-4a37-aa81-ce28f9ceda86 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:57.269440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41bd7f-ce90-4ac4-86fa-e768074b1261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:57.270268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41bd7f-ce90-4ac4-86fa-e768074b1261 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:57.270550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41bd7f-ce90-4ac4-86fa-e768074b1261 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:57.270827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41bd7f-ce90-4ac4-86fa-e768074b1261 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:57.272007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41bd7f-ce90-4ac4-86fa-e768074b1261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:57.273437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41bd7f-ce90-4ac4-86fa-e768074b1261 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:57.275056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 10/18] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:33:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:33:57.280878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41bd7f-ce90-4ac4-86fa-e768074b1261 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-cb506474-01a0-4a37-aa81-ce28f9ceda86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:33:57.286111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:33:57.286986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:33:57.287806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:33:57.287806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:33:57.287806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:33:57.287806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:33:57.287806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 9/19] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:33:57.293107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506474-01a0-4a37-aa81-ce28f9ceda86 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:57.350196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:33:57.361156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:33:57.380196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41bd7f-ce90-4ac4-86fa-e768074b1261 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:57.394931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506474-01a0-4a37-aa81-ce28f9ceda86 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:33:57.414182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531aaac-1c24-4d35-a8c0-20b50f394386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:57.414843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531aaac-1c24-4d35-a8c0-20b50f394386 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:57.415136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531aaac-1c24-4d35-a8c0-20b50f394386 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:57.415406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531aaac-1c24-4d35-a8c0-20b50f394386 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:57.416240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531aaac-1c24-4d35-a8c0-20b50f394386 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.domain.id=default, user.id=d2ffbc8434d34115805dcbc2a72dc671) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:33:57.416549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531aaac-1c24-4d35-a8c0-20b50f394386 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:57.418003 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531aaac-1c24-4d35-a8c0-20b50f394386 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:57.425240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 11/20] 10.222.0.47 () {56 vars in 1322 bytes} [Thu Jul 14 15:33:57 2022] GET /identity/v3/role_assignments?user.id=d2ffbc8434d34115805dcbc2a72dc671&scope.domain.id=default&role.id=abfb60dccca7470295258595a66a5d17 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:33:57.429717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296d7f0-2ee7-418c-b7e8-720ce9a7cb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:33:57.430433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296d7f0-2ee7-418c-b7e8-720ce9a7cb74 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:57.430724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296d7f0-2ee7-418c-b7e8-720ce9a7cb74 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:57.431006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296d7f0-2ee7-418c-b7e8-720ce9a7cb74 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:57.431760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296d7f0-2ee7-418c-b7e8-720ce9a7cb74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:33:57.432073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296d7f0-2ee7-418c-b7e8-720ce9a7cb74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:33:57.433508 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296d7f0-2ee7-418c-b7e8-720ce9a7cb74 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:33:57.454184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 10/21] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:33:59.641787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de7dd9-fd43-40dd-a474-1c425a2d59cf None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:59.642366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de7dd9-fd43-40dd-a474-1c425a2d59cf None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:59.642366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de7dd9-fd43-40dd-a474-1c425a2d59cf None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:59.643747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 12/22] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:33:59.659371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb4d4e-b40e-4ca7-826a-7c578edaefd0 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:59.660211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb4d4e-b40e-4ca7-826a-7c578edaefd0 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:59.660638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb4d4e-b40e-4ca7-826a-7c578edaefd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:59.687024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-27fb4d4e-b40e-4ca7-826a-7c578edaefd0 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:33:59.688650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-27fb4d4e-b40e-4ca7-826a-7c578edaefd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:33:59.780699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 11/23] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:33:59.811497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de7dd9-fd43-40dd-a474-1c425a2d59cf None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:33:59.850299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201ff0c-443d-4140-895f-b476ff24eb4f None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:59.851343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201ff0c-443d-4140-895f-b476ff24eb4f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:59.852202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201ff0c-443d-4140-895f-b476ff24eb4f None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:59.853739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 12/24] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:33:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:33:59.859683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518476ad-1f7f-4012-bbdd-51fff9485c99 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:33:59.860144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518476ad-1f7f-4012-bbdd-51fff9485c99 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:33:59.860567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518476ad-1f7f-4012-bbdd-51fff9485c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:33:59.888027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-518476ad-1f7f-4012-bbdd-51fff9485c99 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:33:59.898638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-518476ad-1f7f-4012-bbdd-51fff9485c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:33:59.998718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:00.043476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de7dd9-fd43-40dd-a474-1c425a2d59cf None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:00.071199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 13/25] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:00.083112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28eb5e-aab1-45eb-847d-1fa1868e7fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:00.083922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28eb5e-aab1-45eb-847d-1fa1868e7fa4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.084209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28eb5e-aab1-45eb-847d-1fa1868e7fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.084492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28eb5e-aab1-45eb-847d-1fa1868e7fa4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.095405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28eb5e-aab1-45eb-847d-1fa1868e7fa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-5b28eb5e-aab1-45eb-847d-1fa1868e7fa4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:00.103306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:00.104187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:00.104951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 13/26] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:00.112117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518476ad-1f7f-4012-bbdd-51fff9485c99 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:00.113649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28eb5e-aab1-45eb-847d-1fa1868e7fa4 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:00.132363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:00.249677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28eb5e-aab1-45eb-847d-1fa1868e7fa4 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:00.290151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d393cd-8773-4a14-bfe9-c20ea27a50e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:00.291359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d393cd-8773-4a14-bfe9-c20ea27a50e0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.292386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d393cd-8773-4a14-bfe9-c20ea27a50e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.292890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d393cd-8773-4a14-bfe9-c20ea27a50e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.305234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d393cd-8773-4a14-bfe9-c20ea27a50e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:00.307538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d393cd-8773-4a14-bfe9-c20ea27a50e0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-65d393cd-8773-4a14-bfe9-c20ea27a50e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:34:00.329214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:00.330144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:00.330969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:34:00.330969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:34:00.330969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:34:00.330969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:00.333286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 14/27] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:00.335185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21484c-65ee-412c-93b6-3a805b6976f2 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.335680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21484c-65ee-412c-93b6-3a805b6976f2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.336237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21484c-65ee-412c-93b6-3a805b6976f2 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.338077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 15/28] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:00.343539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21484c-65ee-412c-93b6-3a805b6976f2 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:00.354539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518476ad-1f7f-4012-bbdd-51fff9485c99 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:00.394540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ac880-9be1-48c8-b925-fd7822f69b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:00.395236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ac880-9be1-48c8-b925-fd7822f69b25 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.395526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ac880-9be1-48c8-b925-fd7822f69b25 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.395808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ac880-9be1-48c8-b925-fd7822f69b25 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.396587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ac880-9be1-48c8-b925-fd7822f69b25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:00.396906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ac880-9be1-48c8-b925-fd7822f69b25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:00.398423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ac880-9be1-48c8-b925-fd7822f69b25 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:00.413643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 14/29] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:00.415136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83799202-141d-4ab4-9640-b63318831c89 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.415436 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83799202-141d-4ab4-9640-b63318831c89 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.415718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83799202-141d-4ab4-9640-b63318831c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.424937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:00.433934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-83799202-141d-4ab4-9640-b63318831c89 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:00.438994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-83799202-141d-4ab4-9640-b63318831c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:00.469315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:00.477387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21484c-65ee-412c-93b6-3a805b6976f2 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:00.512224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 15/30] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:00.513563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6fca8-c41e-4704-a0e4-573aa6954545 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.514495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6fca8-c41e-4704-a0e4-573aa6954545 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.515470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6fca8-c41e-4704-a0e4-573aa6954545 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.517153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 16/31] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:00.517936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6fca8-c41e-4704-a0e4-573aa6954545 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:00.526618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1e1ae-6454-49e2-a923-ca9a769bec56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:00.527309 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1e1ae-6454-49e2-a923-ca9a769bec56 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.527598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1e1ae-6454-49e2-a923-ca9a769bec56 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.527884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1e1ae-6454-49e2-a923-ca9a769bec56 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.528738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1e1ae-6454-49e2-a923-ca9a769bec56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:00.529063 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1e1ae-6454-49e2-a923-ca9a769bec56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:00.530639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1e1ae-6454-49e2-a923-ca9a769bec56 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:00.565552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 16/32] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:00.568107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db48ed-a11b-4361-8dff-7e01c34a1655 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.568408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db48ed-a11b-4361-8dff-7e01c34a1655 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.568690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db48ed-a11b-4361-8dff-7e01c34a1655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.587565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-24db48ed-a11b-4361-8dff-7e01c34a1655 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:00.588781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-24db48ed-a11b-4361-8dff-7e01c34a1655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:00.626159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6fca8-c41e-4704-a0e4-573aa6954545 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:00.661914 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 17/33] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:00.662780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591c637-51f7-4a20-9037-1166a3034fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:00.663394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db48ed-a11b-4361-8dff-7e01c34a1655 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:00.663666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591c637-51f7-4a20-9037-1166a3034fcd None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.663996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591c637-51f7-4a20-9037-1166a3034fcd None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.664271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591c637-51f7-4a20-9037-1166a3034fcd None admin] PATH_INFO: `/v3/users/d2ffbc8434d34115805dcbc2a72dc671` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.666000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591c637-51f7-4a20-9037-1166a3034fcd None admin] RBAC: Authorizing `identity:get_user(user_id=d2ffbc8434d34115805dcbc2a72dc671)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:00.667527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591c637-51f7-4a20-9037-1166a3034fcd None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:00.669566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 17/34] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:00 2022] GET /identity/v3/users/d2ffbc8434d34115805dcbc2a72dc671 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:00.670576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591c637-51f7-4a20-9037-1166a3034fcd None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:00.708563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:00.714237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:00.771139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db48ed-a11b-4361-8dff-7e01c34a1655 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:00.776307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591c637-51f7-4a20-9037-1166a3034fcd None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:00.809097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c443-bd7b-47c4-b43f-cc417f5cb304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:00.809784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c443-bd7b-47c4-b43f-cc417f5cb304 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.810181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c443-bd7b-47c4-b43f-cc417f5cb304 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.810401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74c443-bd7b-47c4-b43f-cc417f5cb304 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.811221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74c443-bd7b-47c4-b43f-cc417f5cb304 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:00.812312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab0f94-86aa-4b8b-87e0-54dad7196e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:00.812620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74c443-bd7b-47c4-b43f-cc417f5cb304 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:00.813068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab0f94-86aa-4b8b-87e0-54dad7196e3b None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:00.813329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab0f94-86aa-4b8b-87e0-54dad7196e3b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:00.813618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab0f94-86aa-4b8b-87e0-54dad7196e3b None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:00.814414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab0f94-86aa-4b8b-87e0-54dad7196e3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:00.815905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab0f94-86aa-4b8b-87e0-54dad7196e3b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:01.007672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 18/35] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 14 15:34:00 2022] POST /identity/v3/roles => generated 219 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:01.015038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c443-bd7b-47c4-b43f-cc417f5cb304 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:01.055140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 18/36] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:34:00 2022] POST /identity/v3/projects => generated 301 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:01.056176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab0f94-86aa-4b8b-87e0-54dad7196e3b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:01.238278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74c443-bd7b-47c4-b43f-cc417f5cb304 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:01.243780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:01.248508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:01.263900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab0f94-86aa-4b8b-87e0-54dad7196e3b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:01.301763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e268fb2-3725-4115-9363-9ee4a004e584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:01.302593 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e268fb2-3725-4115-9363-9ee4a004e584 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:01.302881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e268fb2-3725-4115-9363-9ee4a004e584 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:01.303161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e268fb2-3725-4115-9363-9ee4a004e584 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:01.306786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e268fb2-3725-4115-9363-9ee4a004e584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:01.308407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e268fb2-3725-4115-9363-9ee4a004e584 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:01.310361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 19/37] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:01.316652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e268fb2-3725-4115-9363-9ee4a004e584 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:01.337289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf25b6-b78a-4c45-90e5-945f683c3a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:01.338018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf25b6-b78a-4c45-90e5-945f683c3a44 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:01.338310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf25b6-b78a-4c45-90e5-945f683c3a44 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:01.338587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf25b6-b78a-4c45-90e5-945f683c3a44 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:01.339376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf25b6-b78a-4c45-90e5-945f683c3a44 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:01.340939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf25b6-b78a-4c45-90e5-945f683c3a44 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:01.445980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e268fb2-3725-4115-9363-9ee4a004e584 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:01.462500 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 19/38] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 14 15:34:01 2022] POST /identity/v3/roles => generated 225 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:01.463916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac8d04-a4d5-4459-bbff-a6ff842b4c13 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:01.464202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac8d04-a4d5-4459-bbff-a6ff842b4c13 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:01.464487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac8d04-a4d5-4459-bbff-a6ff842b4c13 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:01.468021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 20/39] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:01.475228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b512-6809-455c-8572-fc8d8f372bf9 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:01.475518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b512-6809-455c-8572-fc8d8f372bf9 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:01.475945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827b512-6809-455c-8572-fc8d8f372bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:01.492715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8f9d9-9d69-4f9f-9b42-1c10941387e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:01.493908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8f9d9-9d69-4f9f-9b42-1c10941387e3 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:01.494356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8f9d9-9d69-4f9f-9b42-1c10941387e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:01.494785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8f9d9-9d69-4f9f-9b42-1c10941387e3 None admin] PATH_INFO: `/v3/domains/default/users/d2ffbc8434d34115805dcbc2a72dc671/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:01.497282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8f9d9-9d69-4f9f-9b42-1c10941387e3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d2ffbc8434d34115805dcbc2a72dc671, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:01.499050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-7827b512-6809-455c-8572-fc8d8f372bf9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:01.499719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8f9d9-9d69-4f9f-9b42-1c10941387e3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:01.500670 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7827b512-6809-455c-8572-fc8d8f372bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:01.580527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 21/40] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:01.580527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a7a60-e563-4fde-a9bb-738a52ddf7d5 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:01.580527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a7a60-e563-4fde-a9bb-738a52ddf7d5 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:01.580527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a7a60-e563-4fde-a9bb-738a52ddf7d5 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:01.580527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 22/41] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:01.585487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb13ba-1498-47ea-8b04-f2a5e85614ae None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:01.585783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb13ba-1498-47ea-8b04-f2a5e85614ae None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:01.586102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb13ba-1498-47ea-8b04-f2a5e85614ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:01.603791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-04fb13ba-1498-47ea-8b04-f2a5e85614ae None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:01.605044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-04fb13ba-1498-47ea-8b04-f2a5e85614ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:01.651765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 20/42] 10.222.0.47 () {58 vars in 1361 bytes} [Thu Jul 14 15:34:01 2022] PUT /identity/v3/domains/default/users/d2ffbc8434d34115805dcbc2a72dc671/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:01.652635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8f9d9-9d69-4f9f-9b42-1c10941387e3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:01.683594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 23/43] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:01.707147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb13ba-1498-47ea-8b04-f2a5e85614ae None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:01.720539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:01.808800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:01.829056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb13ba-1498-47ea-8b04-f2a5e85614ae None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:01.872265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58155c66-99c6-49af-8290-54e2538af92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:01.873187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58155c66-99c6-49af-8290-54e2538af92a None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:01.873488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58155c66-99c6-49af-8290-54e2538af92a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:01.873778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58155c66-99c6-49af-8290-54e2538af92a None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:01.874765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58155c66-99c6-49af-8290-54e2538af92a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:01.876187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58155c66-99c6-49af-8290-54e2538af92a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:01.885155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8f9d9-9d69-4f9f-9b42-1c10941387e3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:01.920805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133d056-f32f-4de0-a412-e3da250abea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:01.921642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133d056-f32f-4de0-a412-e3da250abea6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:01.921969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133d056-f32f-4de0-a412-e3da250abea6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:01.922251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133d056-f32f-4de0-a412-e3da250abea6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:01.923434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133d056-f32f-4de0-a412-e3da250abea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:01.924873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133d056-f32f-4de0-a412-e3da250abea6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:01.926523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 21/44] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:01.931862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133d056-f32f-4de0-a412-e3da250abea6 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:01.980046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 24/45] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 14 15:34:01 2022] POST /identity/v3/roles => generated 223 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:34:02.061298 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133d056-f32f-4de0-a412-e3da250abea6 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:02.063118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:02.101180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ab0f1-9c03-4261-83ae-7af4f9ef1f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:02.101926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ab0f1-9c03-4261-83ae-7af4f9ef1f50 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:02.102209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ab0f1-9c03-4261-83ae-7af4f9ef1f50 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:02.102489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ab0f1-9c03-4261-83ae-7af4f9ef1f50 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:02.103223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ab0f1-9c03-4261-83ae-7af4f9ef1f50 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:02.104700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ab0f1-9c03-4261-83ae-7af4f9ef1f50 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:02.184024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 22/46] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:34:01 2022] POST /identity/v3/projects => generated 312 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:34:03.710863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27067b-33c1-4a9c-ba3e-d2a159ca3213 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:03.711409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27067b-33c1-4a9c-ba3e-d2a159ca3213 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:03.711495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27067b-33c1-4a9c-ba3e-d2a159ca3213 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:03.712956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 25/47] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:03.718116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013391-51d7-414e-ab2e-324bbaf8aeac None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:03.718491 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013391-51d7-414e-ab2e-324bbaf8aeac None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:03.718833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013391-51d7-414e-ab2e-324bbaf8aeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:03.736394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c7013391-51d7-414e-ab2e-324bbaf8aeac None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:03.737957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c7013391-51d7-414e-ab2e-324bbaf8aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:03.803935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 23/48] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:03.827703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27067b-33c1-4a9c-ba3e-d2a159ca3213 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:03.897055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:03.932079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27067b-33c1-4a9c-ba3e-d2a159ca3213 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:03.966107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857394a2-46f6-4e62-aea1-d78a951a96b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:03.967370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857394a2-46f6-4e62-aea1-d78a951a96b3 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:03.967655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857394a2-46f6-4e62-aea1-d78a951a96b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:03.967933 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857394a2-46f6-4e62-aea1-d78a951a96b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:03.979547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857394a2-46f6-4e62-aea1-d78a951a96b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-857394a2-46f6-4e62-aea1-d78a951a96b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:03.988829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:03.989900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:03.990668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 26/49] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:03.997480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013391-51d7-414e-ab2e-324bbaf8aeac None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:04.081577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:04.097342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013391-51d7-414e-ab2e-324bbaf8aeac None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:04.130832 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a07eed-24f8-4a00-a4dc-a0a4a5aeeb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:04.131515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a07eed-24f8-4a00-a4dc-a0a4a5aeeb4e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:04.131803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a07eed-24f8-4a00-a4dc-a0a4a5aeeb4e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:04.132086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a07eed-24f8-4a00-a4dc-a0a4a5aeeb4e None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:04.132848 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a07eed-24f8-4a00-a4dc-a0a4a5aeeb4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:04.133160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a07eed-24f8-4a00-a4dc-a0a4a5aeeb4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:04.134580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a07eed-24f8-4a00-a4dc-a0a4a5aeeb4e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:04.142757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 24/50] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:04.148387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857394a2-46f6-4e62-aea1-d78a951a96b3 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:04.206894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d0962-a775-423b-9640-4d3b1ffb0be7 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:04.207189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d0962-a775-423b-9640-4d3b1ffb0be7 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:04.207472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d0962-a775-423b-9640-4d3b1ffb0be7 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:04.208857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 25/51] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:34:04.213843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea302609-d405-46b3-aa5b-41631756c007 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:04.214170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea302609-d405-46b3-aa5b-41631756c007 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:04.214459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea302609-d405-46b3-aa5b-41631756c007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:04.231418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ea302609-d405-46b3-aa5b-41631756c007 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:04.233157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ea302609-d405-46b3-aa5b-41631756c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:04.250102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857394a2-46f6-4e62-aea1-d78a951a96b3 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:04.276071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:04.283616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947fde6-b0ab-41c4-9e97-dd2e9918c786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:04.286357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947fde6-b0ab-41c4-9e97-dd2e9918c786 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:04.286357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947fde6-b0ab-41c4-9e97-dd2e9918c786 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:04.286357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947fde6-b0ab-41c4-9e97-dd2e9918c786 None admin] PATH_INFO: `/v3/users/d2ffbc8434d34115805dcbc2a72dc671` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:04.286357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947fde6-b0ab-41c4-9e97-dd2e9918c786 None admin] RBAC: Authorizing `identity:get_user(user_id=d2ffbc8434d34115805dcbc2a72dc671)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:04.287664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947fde6-b0ab-41c4-9e97-dd2e9918c786 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:04.289292 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 27/52] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:04 2022] GET /identity/v3/users/d2ffbc8434d34115805dcbc2a72dc671 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:04.294773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947fde6-b0ab-41c4-9e97-dd2e9918c786 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:04.299700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 26/53] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:04.322713 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea302609-d405-46b3-aa5b-41631756c007 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:04.400500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:04.425104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea302609-d405-46b3-aa5b-41631756c007 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:04.462058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e69b9d-2a4d-45f9-8612-ba482e2ba758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:04.462878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e69b9d-2a4d-45f9-8612-ba482e2ba758 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:04.463163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e69b9d-2a4d-45f9-8612-ba482e2ba758 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:04.463439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e69b9d-2a4d-45f9-8612-ba482e2ba758 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:04.464647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e69b9d-2a4d-45f9-8612-ba482e2ba758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:04.466149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e69b9d-2a4d-45f9-8612-ba482e2ba758 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:04.467911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 27/54] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:04.473583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e69b9d-2a4d-45f9-8612-ba482e2ba758 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:04.502667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947fde6-b0ab-41c4-9e97-dd2e9918c786 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:04.536533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c54aa-f8bb-4392-8e03-3ad5121ecc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:04.537325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c54aa-f8bb-4392-8e03-3ad5121ecc83 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:04.537613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c54aa-f8bb-4392-8e03-3ad5121ecc83 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:04.537918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c54aa-f8bb-4392-8e03-3ad5121ecc83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:04.539093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c54aa-f8bb-4392-8e03-3ad5121ecc83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:04.540560 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c54aa-f8bb-4392-8e03-3ad5121ecc83 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:04.542203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 28/55] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:04.548934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c54aa-f8bb-4392-8e03-3ad5121ecc83 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:04.572764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e69b9d-2a4d-45f9-8612-ba482e2ba758 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:04.605986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058383e-0b00-444c-a25f-7ed3589dd2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:04.606650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058383e-0b00-444c-a25f-7ed3589dd2c6 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:04.606944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058383e-0b00-444c-a25f-7ed3589dd2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:04.607748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058383e-0b00-444c-a25f-7ed3589dd2c6 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:04.608478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058383e-0b00-444c-a25f-7ed3589dd2c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:04.609939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058383e-0b00-444c-a25f-7ed3589dd2c6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:04.611137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:04.624228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:04.648892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c54aa-f8bb-4392-8e03-3ad5121ecc83 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:04.682274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8a3fe-b1d1-44df-9f27-b08bbd64a8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:04.682940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8a3fe-b1d1-44df-9f27-b08bbd64a8dc None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:04.683231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8a3fe-b1d1-44df-9f27-b08bbd64a8dc None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:04.683518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8a3fe-b1d1-44df-9f27-b08bbd64a8dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:04.684362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8a3fe-b1d1-44df-9f27-b08bbd64a8dc None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.domain.id=default, user.id=d2ffbc8434d34115805dcbc2a72dc671) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:04.684681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8a3fe-b1d1-44df-9f27-b08bbd64a8dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:04.686180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac8a3fe-b1d1-44df-9f27-b08bbd64a8dc None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:04.695340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 29/56] 10.222.0.47 () {56 vars in 1322 bytes} [Thu Jul 14 15:34:04 2022] GET /identity/v3/role_assignments?user.id=d2ffbc8434d34115805dcbc2a72dc671&scope.domain.id=default&role.id=abfb60dccca7470295258595a66a5d17 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:04.706108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 28/57] 10.222.0.47 () {60 vars in 1140 bytes} [Thu Jul 14 15:34:04 2022] POST /identity/v3/projects => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:34:06.736846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7e03-836a-4037-bb6a-6af25363f4c8 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:06.737361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7e03-836a-4037-bb6a-6af25363f4c8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:06.737444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7e03-836a-4037-bb6a-6af25363f4c8 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:06.738827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 30/58] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:06.744010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc6f8-28d6-4dad-8d0c-f94dd1b69770 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:06.744398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc6f8-28d6-4dad-8d0c-f94dd1b69770 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:06.744584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc6f8-28d6-4dad-8d0c-f94dd1b69770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:06.760941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7e7dc6f8-28d6-4dad-8d0c-f94dd1b69770 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:06.762188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7e7dc6f8-28d6-4dad-8d0c-f94dd1b69770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:06.826741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 29/59] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:06.851150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7e03-836a-4037-bb6a-6af25363f4c8 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:06.952195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7e03-836a-4037-bb6a-6af25363f4c8 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:06.978179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:06.986251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7f3bb-7ea0-4436-a6a6-4e0e9c7e5ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:06.990174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7f3bb-7ea0-4436-a6a6-4e0e9c7e5ce3 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:06.990174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7f3bb-7ea0-4436-a6a6-4e0e9c7e5ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:06.990174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7f3bb-7ea0-4436-a6a6-4e0e9c7e5ce3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:06.990174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7f3bb-7ea0-4436-a6a6-4e0e9c7e5ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:06.990485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7f3bb-7ea0-4436-a6a6-4e0e9c7e5ce3 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:06.991982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 31/60] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:06.997415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc6f8-28d6-4dad-8d0c-f94dd1b69770 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:07.096711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc6f8-28d6-4dad-8d0c-f94dd1b69770 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:07.122156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:07.130483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b487ff-d220-48fc-a5db-e4b81e89f0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:07.131159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b487ff-d220-48fc-a5db-e4b81e89f0c2 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:07.131448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b487ff-d220-48fc-a5db-e4b81e89f0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:07.131730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b487ff-d220-48fc-a5db-e4b81e89f0c2 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:07.132506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b487ff-d220-48fc-a5db-e4b81e89f0c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:07.134035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b487ff-d220-48fc-a5db-e4b81e89f0c2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:07.264272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 30/61] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:34:06 2022] POST /identity/v3/users => generated 275 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:10.400726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8604a-8525-46c1-8ad2-83b8ea1e08df None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:10.401602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8604a-8525-46c1-8ad2-83b8ea1e08df None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:10.402050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8604a-8525-46c1-8ad2-83b8ea1e08df None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:10.403605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 32/62] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:10.409842 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1c365-91ad-49b0-a614-dd5198672ce1 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:10.410636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1c365-91ad-49b0-a614-dd5198672ce1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:10.411166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1c365-91ad-49b0-a614-dd5198672ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:10.431369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-dca1c365-91ad-49b0-a614-dd5198672ce1 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:10.432792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-dca1c365-91ad-49b0-a614-dd5198672ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:10.472211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74428-2ca5-46fb-9f29-439c62061491 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:10.472720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74428-2ca5-46fb-9f29-439c62061491 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:10.473168 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74428-2ca5-46fb-9f29-439c62061491 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:10.474799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 33/63] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:34:10.479973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c400-23c8-483c-960f-20510a97a1d3 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:10.480429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c400-23c8-483c-960f-20510a97a1d3 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:10.480848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9c400-23c8-483c-960f-20510a97a1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:10.501259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-fbf9c400-23c8-483c-960f-20510a97a1d3 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:10.502805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fbf9c400-23c8-483c-960f-20510a97a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:10.553654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 31/64] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:10.579860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1c365-91ad-49b0-a614-dd5198672ce1 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:10.605518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 34/65] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:10.641095 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9c400-23c8-483c-960f-20510a97a1d3 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:10.716730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1c365-91ad-49b0-a614-dd5198672ce1 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:10.731912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:10.779121 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19afedb-e9c5-4b3d-aa0a-11b93edf7629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:10.780196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19afedb-e9c5-4b3d-aa0a-11b93edf7629 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:10.780484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19afedb-e9c5-4b3d-aa0a-11b93edf7629 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:10.780941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19afedb-e9c5-4b3d-aa0a-11b93edf7629 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:10.803249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9c400-23c8-483c-960f-20510a97a1d3 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:10.804463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19afedb-e9c5-4b3d-aa0a-11b93edf7629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-f19afedb-e9c5-4b3d-aa0a-11b93edf7629 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:10.814372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:10.815249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:10.816012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 32/66] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:10.817412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e68f25-6e41-4e19-be1e-891c2f82a922 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:10.817706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e68f25-6e41-4e19-be1e-891c2f82a922 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:10.822071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e68f25-6e41-4e19-be1e-891c2f82a922 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:10.824951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 33/67] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:10.830737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e68f25-6e41-4e19-be1e-891c2f82a922 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:10.846549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:10.862033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e16db-86f5-49cb-bed8-13dac6e2e585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:10.863651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e16db-86f5-49cb-bed8-13dac6e2e585 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:10.864089 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e16db-86f5-49cb-bed8-13dac6e2e585 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:10.864514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e16db-86f5-49cb-bed8-13dac6e2e585 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:10.879061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e16db-86f5-49cb-bed8-13dac6e2e585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-633e16db-86f5-49cb-bed8-13dac6e2e585 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:10.888182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:10.890272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:10.891057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 35/68] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:10.891686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1fc10-cc14-4e9a-a076-6d1cbb95bc63 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:10.891686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1fc10-cc14-4e9a-a076-6d1cbb95bc63 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:10.892209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1fc10-cc14-4e9a-a076-6d1cbb95bc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:10.914939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-98c1fc10-cc14-4e9a-a076-6d1cbb95bc63 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:10.916139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-98c1fc10-cc14-4e9a-a076-6d1cbb95bc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:10.946877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e68f25-6e41-4e19-be1e-891c2f82a922 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:10.984303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02462ca9-1462-42c1-9613-0fd77816a87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:10.984913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02462ca9-1462-42c1-9613-0fd77816a87d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:10.985201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02462ca9-1462-42c1-9613-0fd77816a87d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:10.985485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02462ca9-1462-42c1-9613-0fd77816a87d None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:10.986267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02462ca9-1462-42c1-9613-0fd77816a87d None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:10.986739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02462ca9-1462-42c1-9613-0fd77816a87d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:10.988117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02462ca9-1462-42c1-9613-0fd77816a87d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:10.994969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 36/69] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:11.002147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea180f-972a-4408-90c7-56c49306b420 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.002147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea180f-972a-4408-90c7-56c49306b420 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.002147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea180f-972a-4408-90c7-56c49306b420 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.002147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 37/70] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:11.002147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea180f-972a-4408-90c7-56c49306b420 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.011481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 34/71] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:11.012980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c50290-d77b-470f-a0f3-29cad5cdb159 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.013281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c50290-d77b-470f-a0f3-29cad5cdb159 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.014140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c50290-d77b-470f-a0f3-29cad5cdb159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.035093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-22c50290-d77b-470f-a0f3-29cad5cdb159 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:11.037307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-22c50290-d77b-470f-a0f3-29cad5cdb159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:11.066904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:11.105334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 35/72] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:11.106201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c50290-d77b-470f-a0f3-29cad5cdb159 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.118529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea180f-972a-4408-90c7-56c49306b420 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:11.136216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:11.155336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b721bac-9de4-46a8-92d7-486a415b4e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:11.156014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b721bac-9de4-46a8-92d7-486a415b4e73 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.156302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b721bac-9de4-46a8-92d7-486a415b4e73 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.156582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b721bac-9de4-46a8-92d7-486a415b4e73 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.157330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b721bac-9de4-46a8-92d7-486a415b4e73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:11.157681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b721bac-9de4-46a8-92d7-486a415b4e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:11.159123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b721bac-9de4-46a8-92d7-486a415b4e73 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:11.167202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 38/73] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:11.168005 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b721bac-9de4-46a8-92d7-486a415b4e73 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.213348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c50290-d77b-470f-a0f3-29cad5cdb159 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:11.246672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696dd93-ea43-45a8-9f76-acbb515c45fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:11.247487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696dd93-ea43-45a8-9f76-acbb515c45fa None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.247833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696dd93-ea43-45a8-9f76-acbb515c45fa None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.248611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696dd93-ea43-45a8-9f76-acbb515c45fa None admin] PATH_INFO: `/v3/users/83ad07150d134e288ea59a065b959f79` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.261518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696dd93-ea43-45a8-9f76-acbb515c45fa None admin] RBAC: Authorizing `identity:get_user(user_id=83ad07150d134e288ea59a065b959f79)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:11.263006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696dd93-ea43-45a8-9f76-acbb515c45fa None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:11.264702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 36/74] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:11 2022] GET /identity/v3/users/83ad07150d134e288ea59a065b959f79 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:11.265929 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ae98-b05a-42e6-b677-a0c96b3fc0e4 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.266231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ae98-b05a-42e6-b677-a0c96b3fc0e4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.266509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f3ae98-b05a-42e6-b677-a0c96b3fc0e4 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.267896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 37/75] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:11.268463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ae98-b05a-42e6-b677-a0c96b3fc0e4 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.271068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b721bac-9de4-46a8-92d7-486a415b4e73 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:11.292211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:11.306369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e114bf0-0ec6-4057-a55a-cddfb14184e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:11.307175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e114bf0-0ec6-4057-a55a-cddfb14184e8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.307464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e114bf0-0ec6-4057-a55a-cddfb14184e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.307745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e114bf0-0ec6-4057-a55a-cddfb14184e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.318812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e114bf0-0ec6-4057-a55a-cddfb14184e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-4e114bf0-0ec6-4057-a55a-cddfb14184e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:11.326525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:11.327388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:11.328157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 39/76] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:11.328865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e114bf0-0ec6-4057-a55a-cddfb14184e8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.367909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:11.371612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f3ae98-b05a-42e6-b677-a0c96b3fc0e4 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:11.406185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f53bd-6654-4272-834d-9b223e1641c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:11.406990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f53bd-6654-4272-834d-9b223e1641c5 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.407272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f53bd-6654-4272-834d-9b223e1641c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.407547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f53bd-6654-4272-834d-9b223e1641c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.417738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f53bd-6654-4272-834d-9b223e1641c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-e21f53bd-6654-4272-834d-9b223e1641c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:11.426338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:11.427261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:11.428286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 38/77] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:11.430182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f53bd-6654-4272-834d-9b223e1641c5 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.436115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e114bf0-0ec6-4057-a55a-cddfb14184e8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:11.470271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97c348-5355-470f-923c-4bc2decf56e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:11.471203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97c348-5355-470f-923c-4bc2decf56e1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.471496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97c348-5355-470f-923c-4bc2decf56e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.471774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97c348-5355-470f-923c-4bc2decf56e1 None admin] PATH_INFO: `/v3/users/d2ffbc8434d34115805dcbc2a72dc671` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.472982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97c348-5355-470f-923c-4bc2decf56e1 None admin] RBAC: Authorizing `identity:get_user(user_id=d2ffbc8434d34115805dcbc2a72dc671)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:11.474502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97c348-5355-470f-923c-4bc2decf56e1 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:11.476119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 40/78] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:11 2022] GET /identity/v3/users/d2ffbc8434d34115805dcbc2a72dc671 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:11.477253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aae043-57cb-44d7-bb18-32e12343f55d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.477541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aae043-57cb-44d7-bb18-32e12343f55d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.477840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aae043-57cb-44d7-bb18-32e12343f55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.494525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d9aae043-57cb-44d7-bb18-32e12343f55d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:11.495720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d9aae043-57cb-44d7-bb18-32e12343f55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:11.515981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:11.534407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f53bd-6654-4272-834d-9b223e1641c5 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:11.562179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 41/79] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:11.563026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aae043-57cb-44d7-bb18-32e12343f55d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.568281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7590b72-70d4-4c92-a6d8-de49eb1c2394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:11.568939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7590b72-70d4-4c92-a6d8-de49eb1c2394 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.569223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7590b72-70d4-4c92-a6d8-de49eb1c2394 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.569499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7590b72-70d4-4c92-a6d8-de49eb1c2394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.611295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:11.636487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7590b72-70d4-4c92-a6d8-de49eb1c2394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:11.638079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7590b72-70d4-4c92-a6d8-de49eb1c2394 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:11.675581 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aae043-57cb-44d7-bb18-32e12343f55d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:11.708637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f168cb-7896-4368-b5a3-aab58383d9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:11.709304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f168cb-7896-4368-b5a3-aab58383d9eb None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.709593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f168cb-7896-4368-b5a3-aab58383d9eb None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.709900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f168cb-7896-4368-b5a3-aab58383d9eb None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.710649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f168cb-7896-4368-b5a3-aab58383d9eb None admin] RBAC: Adding query filter params (name=member) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:11.710957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f168cb-7896-4368-b5a3-aab58383d9eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:11.712352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f168cb-7896-4368-b5a3-aab58383d9eb None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:11.717385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:11.721064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 42/80] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:11.721986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f168cb-7896-4368-b5a3-aab58383d9eb None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.736502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 39/81] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:11.737335 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7590b72-70d4-4c92-a6d8-de49eb1c2394 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.821347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:11.825347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f168cb-7896-4368-b5a3-aab58383d9eb None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:11.836375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7590b72-70d4-4c92-a6d8-de49eb1c2394 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:11.860660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514e7b7-fdec-44f7-8887-8a3ffcf769d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:11.861339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e7b7-fdec-44f7-8887-8a3ffcf769d7 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.861635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e7b7-fdec-44f7-8887-8a3ffcf769d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.861953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514e7b7-fdec-44f7-8887-8a3ffcf769d7 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.862716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e7b7-fdec-44f7-8887-8a3ffcf769d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:11.863028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e7b7-fdec-44f7-8887-8a3ffcf769d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:11.864429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514e7b7-fdec-44f7-8887-8a3ffcf769d7 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:11.870382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd518973-0920-4ad7-a82a-a1411ed1bb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:11.871069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd518973-0920-4ad7-a82a-a1411ed1bb91 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:11.871354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd518973-0920-4ad7-a82a-a1411ed1bb91 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:11.871633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd518973-0920-4ad7-a82a-a1411ed1bb91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:11.872455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 43/82] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:11.873870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514e7b7-fdec-44f7-8887-8a3ffcf769d7 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:11.934449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:11.953361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd518973-0920-4ad7-a82a-a1411ed1bb91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:11.955242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd518973-0920-4ad7-a82a-a1411ed1bb91 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:11.978427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514e7b7-fdec-44f7-8887-8a3ffcf769d7 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.014644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2473f-214a-4e37-9151-d48a78d7fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:12.015444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2473f-214a-4e37-9151-d48a78d7fe6d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:12.015735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2473f-214a-4e37-9151-d48a78d7fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:12.016019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2473f-214a-4e37-9151-d48a78d7fe6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:12.017203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2473f-214a-4e37-9151-d48a78d7fe6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:12.018690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2473f-214a-4e37-9151-d48a78d7fe6d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:12.020310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 44/83] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:12.021320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2473f-214a-4e37-9151-d48a78d7fe6d None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:12.050167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:12.060531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 40/84] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:12.061386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd518973-0920-4ad7-a82a-a1411ed1bb91 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:12.125672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2473f-214a-4e37-9151-d48a78d7fe6d None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.142133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:12.159784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b530a8a-ab15-4ef9-abec-7bf77d1cd042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:12.160002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd518973-0920-4ad7-a82a-a1411ed1bb91 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.160669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b530a8a-ab15-4ef9-abec-7bf77d1cd042 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:12.160959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b530a8a-ab15-4ef9-abec-7bf77d1cd042 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:12.161249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b530a8a-ab15-4ef9-abec-7bf77d1cd042 None admin] PATH_INFO: `/v3/users/83ad07150d134e288ea59a065b959f79` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:12.162488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b530a8a-ab15-4ef9-abec-7bf77d1cd042 None admin] RBAC: Authorizing `identity:get_user(user_id=83ad07150d134e288ea59a065b959f79)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:12.163943 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b530a8a-ab15-4ef9-abec-7bf77d1cd042 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:12.165579 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 45/85] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:12 2022] GET /identity/v3/users/83ad07150d134e288ea59a065b959f79 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:12.166386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b530a8a-ab15-4ef9-abec-7bf77d1cd042 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:12.195702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa12018-dda5-4043-9dd9-ff9d529267a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:12.196572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa12018-dda5-4043-9dd9-ff9d529267a7 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:12.196861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa12018-dda5-4043-9dd9-ff9d529267a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:12.197144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa12018-dda5-4043-9dd9-ff9d529267a7 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:12.198373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa12018-dda5-4043-9dd9-ff9d529267a7 None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:12.199842 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa12018-dda5-4043-9dd9-ff9d529267a7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:12.202216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 41/86] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:12 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:12.202974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa12018-dda5-4043-9dd9-ff9d529267a7 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:12.262493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:12.268821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b530a8a-ab15-4ef9-abec-7bf77d1cd042 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.301190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa12018-dda5-4043-9dd9-ff9d529267a7 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.301963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce048-1513-43ec-ac88-089bff3ee586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:12.302849 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce048-1513-43ec-ac88-089bff3ee586 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:12.303139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce048-1513-43ec-ac88-089bff3ee586 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:12.303422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ce048-1513-43ec-ac88-089bff3ee586 None admin] PATH_INFO: `/v3/users/83ad07150d134e288ea59a065b959f79` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:12.304625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ce048-1513-43ec-ac88-089bff3ee586 None admin] RBAC: Authorizing `identity:get_user(user_id=83ad07150d134e288ea59a065b959f79)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:12.306108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ce048-1513-43ec-ac88-089bff3ee586 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:12.307757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 46/87] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:12 2022] GET /identity/v3/users/83ad07150d134e288ea59a065b959f79 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:12.308534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce048-1513-43ec-ac88-089bff3ee586 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:12.336575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23148c7-d3a6-48a3-9327-06aa7a2a1505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:12.337248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23148c7-d3a6-48a3-9327-06aa7a2a1505 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:12.337532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23148c7-d3a6-48a3-9327-06aa7a2a1505 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:12.337805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23148c7-d3a6-48a3-9327-06aa7a2a1505 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:12.338547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23148c7-d3a6-48a3-9327-06aa7a2a1505 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:12.340014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23148c7-d3a6-48a3-9327-06aa7a2a1505 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:12.409749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ce048-1513-43ec-ac88-089bff3ee586 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.443083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398711d-a206-4caa-bc35-ed0c997369e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:12.443945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398711d-a206-4caa-bc35-ed0c997369e4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:12.444236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398711d-a206-4caa-bc35-ed0c997369e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:12.444515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1398711d-a206-4caa-bc35-ed0c997369e4 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:12.445709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398711d-a206-4caa-bc35-ed0c997369e4 None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:12.447208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1398711d-a206-4caa-bc35-ed0c997369e4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:12.447303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 42/88] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:34:12 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:12.448095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23148c7-d3a6-48a3-9327-06aa7a2a1505 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:12.448898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 47/89] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:12 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:12.455347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398711d-a206-4caa-bc35-ed0c997369e4 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:12.458303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:12.498656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:12.568619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23148c7-d3a6-48a3-9327-06aa7a2a1505 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.580145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1398711d-a206-4caa-bc35-ed0c997369e4 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.610972 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b4740-412b-49c5-a99b-54be462b893d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:12.611639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b4740-412b-49c5-a99b-54be462b893d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:12.611937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b4740-412b-49c5-a99b-54be462b893d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:12.612218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b4740-412b-49c5-a99b-54be462b893d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:12.623881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f600a-c293-4ab0-a413-0c6ede2d9574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:12.624558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f600a-c293-4ab0-a413-0c6ede2d9574 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:12.624854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f600a-c293-4ab0-a413-0c6ede2d9574 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:12.625137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f600a-c293-4ab0-a413-0c6ede2d9574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:12.626011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f600a-c293-4ab0-a413-0c6ede2d9574 None admin] RBAC: Adding query filter params (role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=c89e3b28dc2a41c1822590c6a086be14, user.id=83ad07150d134e288ea59a065b959f79) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:12.626333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f600a-c293-4ab0-a413-0c6ede2d9574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:12.627818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f600a-c293-4ab0-a413-0c6ede2d9574 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:12.634049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 48/90] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:12 2022] GET /identity/v3/role_assignments?user.id=83ad07150d134e288ea59a065b959f79&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:12.634862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f600a-c293-4ab0-a413-0c6ede2d9574 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:12.705486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b4740-412b-49c5-a99b-54be462b893d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:12.705486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b4740-412b-49c5-a99b-54be462b893d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:12.720521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:12.772093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:12.783784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f600a-c293-4ab0-a413-0c6ede2d9574 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.829651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a468c-ef21-45a9-b293-53ff26f64876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:12.830498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a468c-ef21-45a9-b293-53ff26f64876 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:12.830794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a468c-ef21-45a9-b293-53ff26f64876 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:12.831078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a468c-ef21-45a9-b293-53ff26f64876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:12.847779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 43/91] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:12.848708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b4740-412b-49c5-a99b-54be462b893d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:12.913840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a468c-ef21-45a9-b293-53ff26f64876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:12.915351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a468c-ef21-45a9-b293-53ff26f64876 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:12.971991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b4740-412b-49c5-a99b-54be462b893d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:12.980079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:13.007906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf40d15-1473-4440-9238-6aa4eef97190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:13.008683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf40d15-1473-4440-9238-6aa4eef97190 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:13.008877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf40d15-1473-4440-9238-6aa4eef97190 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:13.009147 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf40d15-1473-4440-9238-6aa4eef97190 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:13.010196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf40d15-1473-4440-9238-6aa4eef97190 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=c89e3b28dc2a41c1822590c6a086be14, user.id=d2ffbc8434d34115805dcbc2a72dc671) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:13.010527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf40d15-1473-4440-9238-6aa4eef97190 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:13.011977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf40d15-1473-4440-9238-6aa4eef97190 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:13.018695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 44/92] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:12 2022] GET /identity/v3/role_assignments?user.id=d2ffbc8434d34115805dcbc2a72dc671&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=abfb60dccca7470295258595a66a5d17 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:13.019342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf40d15-1473-4440-9238-6aa4eef97190 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:13.051102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:13.059850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 49/93] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:13.072396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a468c-ef21-45a9-b293-53ff26f64876 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:13.191667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf40d15-1473-4440-9238-6aa4eef97190 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:13.262831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa0d6-1a55-41ab-b85c-8ed823031e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:13.265286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa0d6-1a55-41ab-b85c-8ed823031e38 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:13.265929 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa0d6-1a55-41ab-b85c-8ed823031e38 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:13.267514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fa0d6-1a55-41ab-b85c-8ed823031e38 None admin] PATH_INFO: `/v3/projects/7a13aa03cdbd4db0871947b1fb2e05cf` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:13.274891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fa0d6-1a55-41ab-b85c-8ed823031e38 None admin] RBAC: Authorizing `identity:get_project(project_id=7a13aa03cdbd4db0871947b1fb2e05cf)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:13.277643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9fa0d6-1a55-41ab-b85c-8ed823031e38 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:13.280566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 45/94] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:13 2022] GET /identity/v3/projects/7a13aa03cdbd4db0871947b1fb2e05cf => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:13.283046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a468c-ef21-45a9-b293-53ff26f64876 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:13.291031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa0d6-1a55-41ab-b85c-8ed823031e38 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:13.338880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f74893-50f6-4cbb-b7c1-443306529aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:13.339922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f74893-50f6-4cbb-b7c1-443306529aba None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:13.340207 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:13.340406 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f74893-50f6-4cbb-b7c1-443306529aba None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:13.340794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f74893-50f6-4cbb-b7c1-443306529aba None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:13.350654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f74893-50f6-4cbb-b7c1-443306529aba None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:13.352154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f74893-50f6-4cbb-b7c1-443306529aba None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:13.353965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 50/95] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:13 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:13.360236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f74893-50f6-4cbb-b7c1-443306529aba None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:13.406897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fa0d6-1a55-41ab-b85c-8ed823031e38 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:13.454577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb552f61-660c-4c7d-95d1-33baa9742f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:13.456105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552f61-660c-4c7d-95d1-33baa9742f84 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:13.456645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552f61-660c-4c7d-95d1-33baa9742f84 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:13.457122 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552f61-660c-4c7d-95d1-33baa9742f84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:13.458211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb552f61-660c-4c7d-95d1-33baa9742f84 None admin] RBAC: Adding query filter params (role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=7a13aa03cdbd4db0871947b1fb2e05cf, user.id=83ad07150d134e288ea59a065b959f79) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:13.458728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb552f61-660c-4c7d-95d1-33baa9742f84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:13.460417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb552f61-660c-4c7d-95d1-33baa9742f84 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:13.465922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:13.467371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 46/96] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:13 2022] GET /identity/v3/role_assignments?user.id=83ad07150d134e288ea59a065b959f79&scope.project.id=7a13aa03cdbd4db0871947b1fb2e05cf&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:13.522205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f74893-50f6-4cbb-b7c1-443306529aba None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:13.573152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199220a5-2cc9-46e9-85b6-e492869c86eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:13.573842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199220a5-2cc9-46e9-85b6-e492869c86eb None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:13.574143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199220a5-2cc9-46e9-85b6-e492869c86eb None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:13.574429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199220a5-2cc9-46e9-85b6-e492869c86eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:13.575284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199220a5-2cc9-46e9-85b6-e492869c86eb None admin] RBAC: Adding query filter params (role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=c89e3b28dc2a41c1822590c6a086be14, user.id=83ad07150d134e288ea59a065b959f79) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:13.575610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199220a5-2cc9-46e9-85b6-e492869c86eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:13.577114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199220a5-2cc9-46e9-85b6-e492869c86eb None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:13.583352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 51/97] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:13 2022] GET /identity/v3/role_assignments?user.id=83ad07150d134e288ea59a065b959f79&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:15.039431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884e062-4135-4a91-a930-a91aa5613426 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:15.039951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884e062-4135-4a91-a930-a91aa5613426 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:15.040057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884e062-4135-4a91-a930-a91aa5613426 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:15.041872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 47/98] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:15.060029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30a5a9-6f9f-4a28-87c0-c3d05a24ff31 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:15.060579 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30a5a9-6f9f-4a28-87c0-c3d05a24ff31 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:15.060653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30a5a9-6f9f-4a28-87c0-c3d05a24ff31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:15.090006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6c30a5a9-6f9f-4a28-87c0-c3d05a24ff31 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:15.091243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6c30a5a9-6f9f-4a28-87c0-c3d05a24ff31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:15.184543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 52/99] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:15.231269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884e062-4135-4a91-a930-a91aa5613426 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:15.354745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884e062-4135-4a91-a930-a91aa5613426 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:15.356507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:15.401440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980b2d2-c771-460f-94fe-37d0f1e4dae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:15.402272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980b2d2-c771-460f-94fe-37d0f1e4dae1 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:15.402554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980b2d2-c771-460f-94fe-37d0f1e4dae1 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:15.402828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980b2d2-c771-460f-94fe-37d0f1e4dae1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:15.418693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980b2d2-c771-460f-94fe-37d0f1e4dae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-b980b2d2-c771-460f-94fe-37d0f1e4dae1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:15.427105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:15.427967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:15.428743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 48/100] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:15.438930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30a5a9-6f9f-4a28-87c0-c3d05a24ff31 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:15.515569 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a657aff-8216-4bce-befc-ffbbbf56eb5d None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:15.515880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a657aff-8216-4bce-befc-ffbbbf56eb5d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:15.516138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a657aff-8216-4bce-befc-ffbbbf56eb5d None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:15.517750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 49/101] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:15.522875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b18a05-d1d4-4739-87b4-bc9fe88d22cf None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:15.523163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b18a05-d1d4-4739-87b4-bc9fe88d22cf None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:15.523443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b18a05-d1d4-4739-87b4-bc9fe88d22cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:15.534851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:15.554859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f4b18a05-d1d4-4739-87b4-bc9fe88d22cf None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:15.556085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f4b18a05-d1d4-4739-87b4-bc9fe88d22cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:15.581440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30a5a9-6f9f-4a28-87c0-c3d05a24ff31 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:15.625881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0de391-0d01-408f-9468-89bf08bc6a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:15.626493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0de391-0d01-408f-9468-89bf08bc6a29 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:15.626779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0de391-0d01-408f-9468-89bf08bc6a29 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:15.627057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0de391-0d01-408f-9468-89bf08bc6a29 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:15.627815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0de391-0d01-408f-9468-89bf08bc6a29 None admin] RBAC: Adding query filter params (name=member) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:15.628126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0de391-0d01-408f-9468-89bf08bc6a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:15.629528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0de391-0d01-408f-9468-89bf08bc6a29 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:15.647172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 53/102] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:15.652953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88af12a-6cd2-456e-9b2f-c14ab6b71ec1 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:15.653510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88af12a-6cd2-456e-9b2f-c14ab6b71ec1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:15.654634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88af12a-6cd2-456e-9b2f-c14ab6b71ec1 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:15.656308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 54/103] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:15.657266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88af12a-6cd2-456e-9b2f-c14ab6b71ec1 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:15.666907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 50/104] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:15.668374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b21d0-b008-46d8-a5fa-4be1aa3f6a1e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:15.668670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b21d0-b008-46d8-a5fa-4be1aa3f6a1e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:15.668952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b21d0-b008-46d8-a5fa-4be1aa3f6a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:15.686711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b16b21d0-b008-46d8-a5fa-4be1aa3f6a1e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:15.688393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b16b21d0-b008-46d8-a5fa-4be1aa3f6a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:15.780170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 51/105] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:15.781525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b21d0-b008-46d8-a5fa-4be1aa3f6a1e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:15.827880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:15.869037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88af12a-6cd2-456e-9b2f-c14ab6b71ec1 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:15.908137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:15.939644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a232f-689e-49de-98d5-8ab18b731b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:15.940877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a232f-689e-49de-98d5-8ab18b731b97 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:15.941627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a232f-689e-49de-98d5-8ab18b731b97 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:15.942129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a232f-689e-49de-98d5-8ab18b731b97 None admin] PATH_INFO: `/v3/users/83ad07150d134e288ea59a065b959f79` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:15.944607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a232f-689e-49de-98d5-8ab18b731b97 None admin] RBAC: Authorizing `identity:get_user(user_id=83ad07150d134e288ea59a065b959f79)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:15.946445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a232f-689e-49de-98d5-8ab18b731b97 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:15.948480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 55/106] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:15 2022] GET /identity/v3/users/83ad07150d134e288ea59a065b959f79 => generated 275 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:15.950832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a232f-689e-49de-98d5-8ab18b731b97 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:15.968517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b21d0-b008-46d8-a5fa-4be1aa3f6a1e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:16.038344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1614999-c27f-453b-9e42-b0ee8e0fc81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:16.039372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1614999-c27f-453b-9e42-b0ee8e0fc81b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.039481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1614999-c27f-453b-9e42-b0ee8e0fc81b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.039960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1614999-c27f-453b-9e42-b0ee8e0fc81b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.065582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1614999-c27f-453b-9e42-b0ee8e0fc81b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-f1614999-c27f-453b-9e42-b0ee8e0fc81b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:16.081919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:16.083824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:16.084599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:16.086869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 52/107] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:16.089316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1614999-c27f-453b-9e42-b0ee8e0fc81b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:16.185791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a232f-689e-49de-98d5-8ab18b731b97 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:16.256176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8837fb-bd89-4e3c-b23b-6bce418c4b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:16.257361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8837fb-bd89-4e3c-b23b-6bce418c4b94 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.258593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8837fb-bd89-4e3c-b23b-6bce418c4b94 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.259571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8837fb-bd89-4e3c-b23b-6bce418c4b94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.262828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8837fb-bd89-4e3c-b23b-6bce418c4b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:16.264510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8837fb-bd89-4e3c-b23b-6bce418c4b94 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:16.267107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 56/108] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:16.269060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8837fb-bd89-4e3c-b23b-6bce418c4b94 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:16.300302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:16.336437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1614999-c27f-453b-9e42-b0ee8e0fc81b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:16.386208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:16.402431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506bbb3-3d5d-4206-9ec3-51b6be63e85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:16.403643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506bbb3-3d5d-4206-9ec3-51b6be63e85b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.404639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506bbb3-3d5d-4206-9ec3-51b6be63e85b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.405066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506bbb3-3d5d-4206-9ec3-51b6be63e85b None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.406633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506bbb3-3d5d-4206-9ec3-51b6be63e85b None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:16.409268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506bbb3-3d5d-4206-9ec3-51b6be63e85b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:16.411498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 53/109] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:16 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:16.412417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506bbb3-3d5d-4206-9ec3-51b6be63e85b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:16.443330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8837fb-bd89-4e3c-b23b-6bce418c4b94 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:16.481284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240db87e-1b9d-42e3-9789-2991d81cd451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:16.481996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240db87e-1b9d-42e3-9789-2991d81cd451 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.482315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240db87e-1b9d-42e3-9789-2991d81cd451 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.482699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240db87e-1b9d-42e3-9789-2991d81cd451 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.483462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240db87e-1b9d-42e3-9789-2991d81cd451 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:16.483771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240db87e-1b9d-42e3-9789-2991d81cd451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:16.485163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240db87e-1b9d-42e3-9789-2991d81cd451 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:16.494923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 57/110] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:16.495742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240db87e-1b9d-42e3-9789-2991d81cd451 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:16.537217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506bbb3-3d5d-4206-9ec3-51b6be63e85b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:16.574954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cba339-fc90-4dff-891b-3ca41caf094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:16.575630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cba339-fc90-4dff-891b-3ca41caf094a None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.576456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cba339-fc90-4dff-891b-3ca41caf094a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.576741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cba339-fc90-4dff-891b-3ca41caf094a None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.577470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cba339-fc90-4dff-891b-3ca41caf094a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:16.578994 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cba339-fc90-4dff-891b-3ca41caf094a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:16.604079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240db87e-1b9d-42e3-9789-2991d81cd451 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:16.620792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:16.637143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983dc81-995a-4b8d-901e-1fe74de80df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:16.638156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983dc81-995a-4b8d-901e-1fe74de80df6 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.638990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983dc81-995a-4b8d-901e-1fe74de80df6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.639276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983dc81-995a-4b8d-901e-1fe74de80df6 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14/users/83ad07150d134e288ea59a065b959f79/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.641236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983dc81-995a-4b8d-901e-1fe74de80df6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c89e3b28dc2a41c1822590c6a086be14, user_id=83ad07150d134e288ea59a065b959f79, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:16.642718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983dc81-995a-4b8d-901e-1fe74de80df6 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:16.674139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 54/111] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:34:16 2022] POST /identity/v3/projects => generated 302 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:16.676591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60acb8-535c-4320-88d7-76ba6f865045 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.677798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60acb8-535c-4320-88d7-76ba6f865045 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.677798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60acb8-535c-4320-88d7-76ba6f865045 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.678891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 55/112] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:16.679540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60acb8-535c-4320-88d7-76ba6f865045 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:16.693165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:16.796521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60acb8-535c-4320-88d7-76ba6f865045 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:16.811453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 58/113] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:34:16 2022] PUT /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14/users/83ad07150d134e288ea59a065b959f79/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:16.813485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e732c3-9197-4586-b0ef-84ef6b1ac344 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.814038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e732c3-9197-4586-b0ef-84ef6b1ac344 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.814530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e732c3-9197-4586-b0ef-84ef6b1ac344 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.816828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 59/114] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:16.820737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19605b-2a3f-49c7-b7ea-f6c83c271e89 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.821351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19605b-2a3f-49c7-b7ea-f6c83c271e89 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.821857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19605b-2a3f-49c7-b7ea-f6c83c271e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.847593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5a19605b-2a3f-49c7-b7ea-f6c83c271e89 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:16.847593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5a19605b-2a3f-49c7-b7ea-f6c83c271e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:16.853628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73fd7f-4177-4f70-aa90-4ec9f47b680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:16.854592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73fd7f-4177-4f70-aa90-4ec9f47b680f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.854875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73fd7f-4177-4f70-aa90-4ec9f47b680f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.855152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73fd7f-4177-4f70-aa90-4ec9f47b680f None admin] PATH_INFO: `/v3/users/0b56f8efe53744f3ae9bc5ec439bbfcd` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.876444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73fd7f-4177-4f70-aa90-4ec9f47b680f None admin] RBAC: Authorizing `identity:get_user(user_id=0b56f8efe53744f3ae9bc5ec439bbfcd)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:16.879301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73fd7f-4177-4f70-aa90-4ec9f47b680f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:16.880259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 56/115] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:16 2022] GET /identity/v3/users/0b56f8efe53744f3ae9bc5ec439bbfcd => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:16.881529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd586-8f21-40cd-8ab4-62a592220415 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.881810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd586-8f21-40cd-8ab4-62a592220415 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.882893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9cd586-8f21-40cd-8ab4-62a592220415 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.884351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 57/116] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:16.885550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dc363-0592-41d9-bc72-62553e57e829 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:16.885991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dc363-0592-41d9-bc72-62553e57e829 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:16.886275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dc363-0592-41d9-bc72-62553e57e829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:16.905738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-301dc363-0592-41d9-bc72-62553e57e829 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:16.909371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-301dc363-0592-41d9-bc72-62553e57e829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:16.931978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 60/117] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:16.933052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19605b-2a3f-49c7-b7ea-f6c83c271e89 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:16.996345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:17.008554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 58/118] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:17.015792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc90a1-7513-4724-800d-6ac126219251 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:17.015792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc90a1-7513-4724-800d-6ac126219251 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:17.015792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc90a1-7513-4724-800d-6ac126219251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:17.029489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-23fc90a1-7513-4724-800d-6ac126219251 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:17.030737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-23fc90a1-7513-4724-800d-6ac126219251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:17.071134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19605b-2a3f-49c7-b7ea-f6c83c271e89 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:17.086503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:17.113478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073d19-be5c-4f20-94ee-f622d09ec0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:17.114194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073d19-be5c-4f20-94ee-f622d09ec0a3 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:17.114489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073d19-be5c-4f20-94ee-f622d09ec0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:17.114767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073d19-be5c-4f20-94ee-f622d09ec0a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:17.128848 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 59/119] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:17.130730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc90a1-7513-4724-800d-6ac126219251 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:17.210028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6073d19-be5c-4f20-94ee-f622d09ec0a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:17.212297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6073d19-be5c-4f20-94ee-f622d09ec0a3 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:17.265772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc90a1-7513-4724-800d-6ac126219251 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:17.293510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:17.313934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ada27-19ad-418b-afce-c994e05be1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:17.314674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ada27-19ad-418b-afce-c994e05be1d8 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:17.315962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ada27-19ad-418b-afce-c994e05be1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:17.315962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ada27-19ad-418b-afce-c994e05be1d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:17.326194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 61/120] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:17.327378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073d19-be5c-4f20-94ee-f622d09ec0a3 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:17.329339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ada27-19ad-418b-afce-c994e05be1d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:17.339411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-ce3ada27-19ad-418b-afce-c994e05be1d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:17.340289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:17.341142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:17.341992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 60/121] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:17.343215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ada27-19ad-418b-afce-c994e05be1d8 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:17.444218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073d19-be5c-4f20-94ee-f622d09ec0a3 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:17.480713 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ada27-19ad-418b-afce-c994e05be1d8 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:17.488052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73b858-ad87-40ed-91fc-9256898eb64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:17.489095 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73b858-ad87-40ed-91fc-9256898eb64b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:17.489584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73b858-ad87-40ed-91fc-9256898eb64b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:17.490108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73b858-ad87-40ed-91fc-9256898eb64b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:17.501085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73b858-ad87-40ed-91fc-9256898eb64b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-2d73b858-ad87-40ed-91fc-9256898eb64b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:17.509141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:17.510020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:17.510811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:17.519138 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 62/122] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:17.519138 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73b858-ad87-40ed-91fc-9256898eb64b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:17.524467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032c3fd-269d-4220-965d-00b42e401bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:17.525578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032c3fd-269d-4220-965d-00b42e401bca None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:17.526095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032c3fd-269d-4220-965d-00b42e401bca None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:17.526579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032c3fd-269d-4220-965d-00b42e401bca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:17.538614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032c3fd-269d-4220-965d-00b42e401bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-7032c3fd-269d-4220-965d-00b42e401bca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:17.547511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:17.548378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:17.549180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:17.551555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 61/123] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:17.552564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032c3fd-269d-4220-965d-00b42e401bca None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:17.597972 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:17.612070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:17.648062 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73b858-ad87-40ed-91fc-9256898eb64b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:17.678813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032c3fd-269d-4220-965d-00b42e401bca None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:17.685521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df6b92-3422-4a0c-8e68-6c5e888e976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:17.686438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df6b92-3422-4a0c-8e68-6c5e888e976b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:17.686729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df6b92-3422-4a0c-8e68-6c5e888e976b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:17.687015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df6b92-3422-4a0c-8e68-6c5e888e976b None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:17.688224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df6b92-3422-4a0c-8e68-6c5e888e976b None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:17.689688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df6b92-3422-4a0c-8e68-6c5e888e976b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:17.692116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 63/124] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:17 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:17.693018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df6b92-3422-4a0c-8e68-6c5e888e976b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:17.724711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee1e01-53e4-4336-8a22-ca6f779e0d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:17.725959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee1e01-53e4-4336-8a22-ca6f779e0d3f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:17.726281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee1e01-53e4-4336-8a22-ca6f779e0d3f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:17.726571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee1e01-53e4-4336-8a22-ca6f779e0d3f None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:17.727350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee1e01-53e4-4336-8a22-ca6f779e0d3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:17.727672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee1e01-53e4-4336-8a22-ca6f779e0d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:17.729099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee1e01-53e4-4336-8a22-ca6f779e0d3f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:17.738333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 62/125] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:17.739143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee1e01-53e4-4336-8a22-ca6f779e0d3f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:17.828420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df6b92-3422-4a0c-8e68-6c5e888e976b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:17.851332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee1e01-53e4-4336-8a22-ca6f779e0d3f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:17.860941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:17.867569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b9ae7-c480-469f-8fc9-fe27870008e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:17.868547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b9ae7-c480-469f-8fc9-fe27870008e0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:17.869087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b9ae7-c480-469f-8fc9-fe27870008e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:17.869559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b9ae7-c480-469f-8fc9-fe27870008e0 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:17.870529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b9ae7-c480-469f-8fc9-fe27870008e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:17.870905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b9ae7-c480-469f-8fc9-fe27870008e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:17.876586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b9ae7-c480-469f-8fc9-fe27870008e0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:17.876586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:17.886332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 64/126] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:17.887244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b9ae7-c480-469f-8fc9-fe27870008e0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:17.898356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b998c1-9ec9-418f-be2a-140de82e6c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:17.899062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b998c1-9ec9-418f-be2a-140de82e6c01 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:17.899354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b998c1-9ec9-418f-be2a-140de82e6c01 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:17.899643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b998c1-9ec9-418f-be2a-140de82e6c01 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:17.900476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b998c1-9ec9-418f-be2a-140de82e6c01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:17.900842 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b998c1-9ec9-418f-be2a-140de82e6c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:17.902308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b998c1-9ec9-418f-be2a-140de82e6c01 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:17.912915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 63/127] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:17.913616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b998c1-9ec9-418f-be2a-140de82e6c01 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:17.995039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b9ae7-c480-469f-8fc9-fe27870008e0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.026354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b998c1-9ec9-418f-be2a-140de82e6c01 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.051773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723464d-e53d-48e0-9629-7cbc85f3a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.052862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723464d-e53d-48e0-9629-7cbc85f3a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.053587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723464d-e53d-48e0-9629-7cbc85f3a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.054068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723464d-e53d-48e0-9629-7cbc85f3a8f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.055226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723464d-e53d-48e0-9629-7cbc85f3a8f6 None admin] RBAC: Adding query filter params (role.id=eb50a4dbe6434ab0847618153c619d1a, scope.project.id=c89e3b28dc2a41c1822590c6a086be14, user.id=0b56f8efe53744f3ae9bc5ec439bbfcd) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:18.055749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723464d-e53d-48e0-9629-7cbc85f3a8f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.058203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723464d-e53d-48e0-9629-7cbc85f3a8f6 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:18.066762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 65/128] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:17 2022] GET /identity/v3/role_assignments?user.id=0b56f8efe53744f3ae9bc5ec439bbfcd&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=eb50a4dbe6434ab0847618153c619d1a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:18.067790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723464d-e53d-48e0-9629-7cbc85f3a8f6 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:18.087061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf79c-cac1-47c1-a264-ae016c68a87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.088133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf79c-cac1-47c1-a264-ae016c68a87a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.088459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf79c-cac1-47c1-a264-ae016c68a87a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.088854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cf79c-cac1-47c1-a264-ae016c68a87a None admin] PATH_INFO: `/v3/users/d2ffbc8434d34115805dcbc2a72dc671` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.094832 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cf79c-cac1-47c1-a264-ae016c68a87a None admin] RBAC: Authorizing `identity:get_user(user_id=d2ffbc8434d34115805dcbc2a72dc671)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.096639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cf79c-cac1-47c1-a264-ae016c68a87a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:18.098606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 64/129] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:17 2022] GET /identity/v3/users/d2ffbc8434d34115805dcbc2a72dc671 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:18.100225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf79c-cac1-47c1-a264-ae016c68a87a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:18.182329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:18.186936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723464d-e53d-48e0-9629-7cbc85f3a8f6 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.195291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:18.234386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ea731-f6c2-4fe8-a25a-d13a79003055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.235940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ea731-f6c2-4fe8-a25a-d13a79003055 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.236393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ea731-f6c2-4fe8-a25a-d13a79003055 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.237805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ea731-f6c2-4fe8-a25a-d13a79003055 None admin] PATH_INFO: `/v3/users/83ad07150d134e288ea59a065b959f79` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.238699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ea731-f6c2-4fe8-a25a-d13a79003055 None admin] RBAC: Authorizing `identity:get_user(user_id=83ad07150d134e288ea59a065b959f79)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.242136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ea731-f6c2-4fe8-a25a-d13a79003055 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:18.247169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 66/130] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:18 2022] GET /identity/v3/users/83ad07150d134e288ea59a065b959f79 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:18.249034 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ea731-f6c2-4fe8-a25a-d13a79003055 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:18.319400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cf79c-cac1-47c1-a264-ae016c68a87a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.357485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb9651-6bcb-44e1-b534-0ccc2d3768b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.358739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb9651-6bcb-44e1-b534-0ccc2d3768b3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.360345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb9651-6bcb-44e1-b534-0ccc2d3768b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.362211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb9651-6bcb-44e1-b534-0ccc2d3768b3 None admin] PATH_INFO: `/v3/users/83ad07150d134e288ea59a065b959f79` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.363896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb9651-6bcb-44e1-b534-0ccc2d3768b3 None admin] RBAC: Authorizing `identity:get_user(user_id=83ad07150d134e288ea59a065b959f79)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.366508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb9651-6bcb-44e1-b534-0ccc2d3768b3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:18.371720 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 65/131] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:18 2022] GET /identity/v3/users/83ad07150d134e288ea59a065b959f79 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:18.373967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb9651-6bcb-44e1-b534-0ccc2d3768b3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:18.489512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:18.493587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb9651-6bcb-44e1-b534-0ccc2d3768b3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.527797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ea731-f6c2-4fe8-a25a-d13a79003055 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.532673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cbd2c-c5a0-4d9a-86cd-863f1bc635fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.533841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cbd2c-c5a0-4d9a-86cd-863f1bc635fb None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.534347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cbd2c-c5a0-4d9a-86cd-863f1bc635fb None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.534853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cbd2c-c5a0-4d9a-86cd-863f1bc635fb None admin] PATH_INFO: `/v3/projects/7a13aa03cdbd4db0871947b1fb2e05cf` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.536273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cbd2c-c5a0-4d9a-86cd-863f1bc635fb None admin] RBAC: Authorizing `identity:get_project(project_id=7a13aa03cdbd4db0871947b1fb2e05cf)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.537949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cbd2c-c5a0-4d9a-86cd-863f1bc635fb None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:18.540296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 66/132] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:18 2022] GET /identity/v3/projects/7a13aa03cdbd4db0871947b1fb2e05cf => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:18.541294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cbd2c-c5a0-4d9a-86cd-863f1bc635fb None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:18.586984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:18.594316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe868fe4-077a-45bf-bbb0-0b16f0d25d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.595195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe868fe4-077a-45bf-bbb0-0b16f0d25d45 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.595479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe868fe4-077a-45bf-bbb0-0b16f0d25d45 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.595755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe868fe4-077a-45bf-bbb0-0b16f0d25d45 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.597601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe868fe4-077a-45bf-bbb0-0b16f0d25d45 None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.599109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe868fe4-077a-45bf-bbb0-0b16f0d25d45 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:18.602123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 67/133] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:18 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:18.602872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe868fe4-077a-45bf-bbb0-0b16f0d25d45 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:18.719808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cbd2c-c5a0-4d9a-86cd-863f1bc635fb None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.749735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe868fe4-077a-45bf-bbb0-0b16f0d25d45 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.786994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700df894-046e-4638-a8fc-4c4d0d57db93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.787901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700df894-046e-4638-a8fc-4c4d0d57db93 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.788191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700df894-046e-4638-a8fc-4c4d0d57db93 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.788470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700df894-046e-4638-a8fc-4c4d0d57db93 None admin] PATH_INFO: `/v3/projects/7a13aa03cdbd4db0871947b1fb2e05cf/users/83ad07150d134e288ea59a065b959f79/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.790503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700df894-046e-4638-a8fc-4c4d0d57db93 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a13aa03cdbd4db0871947b1fb2e05cf, user_id=83ad07150d134e288ea59a065b959f79, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.792062 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700df894-046e-4638-a8fc-4c4d0d57db93 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:18.802588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa265ba1-d6b4-4e91-8e64-c56c2c2e39da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.803524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa265ba1-d6b4-4e91-8e64-c56c2c2e39da None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.803918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa265ba1-d6b4-4e91-8e64-c56c2c2e39da None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.804841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa265ba1-d6b4-4e91-8e64-c56c2c2e39da None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.805675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa265ba1-d6b4-4e91-8e64-c56c2c2e39da None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.807774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa265ba1-d6b4-4e91-8e64-c56c2c2e39da None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:18.810190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 67/134] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:18 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:18.811146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa265ba1-d6b4-4e91-8e64-c56c2c2e39da None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:18.853252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:18.910478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 68/135] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:34:18 2022] PUT /identity/v3/projects/7a13aa03cdbd4db0871947b1fb2e05cf/users/83ad07150d134e288ea59a065b959f79/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:18.912606 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700df894-046e-4638-a8fc-4c4d0d57db93 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:18.934800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa265ba1-d6b4-4e91-8e64-c56c2c2e39da None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:18.981419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d2d59-c283-4870-9fb9-a796475094a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:18.982488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2d59-c283-4870-9fb9-a796475094a2 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:18.982773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2d59-c283-4870-9fb9-a796475094a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:18.983180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2d59-c283-4870-9fb9-a796475094a2 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14/users/d2ffbc8434d34115805dcbc2a72dc671/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:18.985498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d2d59-c283-4870-9fb9-a796475094a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c89e3b28dc2a41c1822590c6a086be14, user_id=d2ffbc8434d34115805dcbc2a72dc671, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:18.987068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d2d59-c283-4870-9fb9-a796475094a2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:18.994996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:19.059837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700df894-046e-4638-a8fc-4c4d0d57db93 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:19.077242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 68/136] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:34:18 2022] PUT /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14/users/d2ffbc8434d34115805dcbc2a72dc671/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:19.078721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4cfbf-7086-42bc-a333-faa8c8299edb None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.079137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4cfbf-7086-42bc-a333-faa8c8299edb None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.079355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4cfbf-7086-42bc-a333-faa8c8299edb None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.080999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 69/137] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:19.089020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e564a6-716f-4d8e-8484-38313728bd04 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.089348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e564a6-716f-4d8e-8484-38313728bd04 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.089574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e564a6-716f-4d8e-8484-38313728bd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.115385 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fb67c-bdaa-428f-ab53-750df35fc0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:19.116769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fb67c-bdaa-428f-ab53-750df35fc0e6 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.117280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fb67c-bdaa-428f-ab53-750df35fc0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.117773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fb67c-bdaa-428f-ab53-750df35fc0e6 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14/users/83ad07150d134e288ea59a065b959f79/roles/2806b70c57be44f9b260ff5322bf02fb` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.120479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fb67c-bdaa-428f-ab53-750df35fc0e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c89e3b28dc2a41c1822590c6a086be14, user_id=83ad07150d134e288ea59a065b959f79, role_id=2806b70c57be44f9b260ff5322bf02fb)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:19.122267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d6e564a6-716f-4d8e-8484-38313728bd04 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:19.122544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fb67c-bdaa-428f-ab53-750df35fc0e6 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:19.124197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d6e564a6-716f-4d8e-8484-38313728bd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:19.216105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 70/138] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:19.219711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29dee-a517-4669-afd9-cb9a539e76f2 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.220014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29dee-a517-4669-afd9-cb9a539e76f2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.220300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e29dee-a517-4669-afd9-cb9a539e76f2 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.221845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 71/139] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:19.229361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301b189-5fbc-4933-b09f-0dbc62c1fabc None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.229743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301b189-5fbc-4933-b09f-0dbc62c1fabc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.230036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301b189-5fbc-4933-b09f-0dbc62c1fabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.257770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a301b189-5fbc-4933-b09f-0dbc62c1fabc None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:19.262745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a301b189-5fbc-4933-b09f-0dbc62c1fabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:19.262858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 69/140] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:34:18 2022] PUT /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14/users/83ad07150d134e288ea59a065b959f79/roles/2806b70c57be44f9b260ff5322bf02fb => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:19.262858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fb67c-bdaa-428f-ab53-750df35fc0e6 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:19.286417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:19.361140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 72/141] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:19.391955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301b189-5fbc-4933-b09f-0dbc62c1fabc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:19.423476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fb67c-bdaa-428f-ab53-750df35fc0e6 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:19.451204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:19.471063 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9226d3-e59a-481b-ab8c-21ccc4a1415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:19.472324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9226d3-e59a-481b-ab8c-21ccc4a1415a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.472832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9226d3-e59a-481b-ab8c-21ccc4a1415a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.473360 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9226d3-e59a-481b-ab8c-21ccc4a1415a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.474922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9226d3-e59a-481b-ab8c-21ccc4a1415a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:19.476743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9226d3-e59a-481b-ab8c-21ccc4a1415a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:19.478753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 70/142] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:19.485265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9226d3-e59a-481b-ab8c-21ccc4a1415a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:19.531893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301b189-5fbc-4933-b09f-0dbc62c1fabc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:19.602426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0c338-bded-46b4-be0b-5ddcdfec2c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:19.603337 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0c338-bded-46b4-be0b-5ddcdfec2c5d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.603622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0c338-bded-46b4-be0b-5ddcdfec2c5d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.603898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0c338-bded-46b4-be0b-5ddcdfec2c5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.617366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b0c338-bded-46b4-be0b-5ddcdfec2c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-03b0c338-bded-46b4-be0b-5ddcdfec2c5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:19.626922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:19.627841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:19.629047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 73/143] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:19.634639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0c338-bded-46b4-be0b-5ddcdfec2c5d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:19.642712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9226d3-e59a-481b-ab8c-21ccc4a1415a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:19.659590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:19.691693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd84d2-fba9-43c1-ab23-e15ce1b3784e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:19.692388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd84d2-fba9-43c1-ab23-e15ce1b3784e None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.692678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd84d2-fba9-43c1-ab23-e15ce1b3784e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.693117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd84d2-fba9-43c1-ab23-e15ce1b3784e None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.694185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd84d2-fba9-43c1-ab23-e15ce1b3784e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:19.695667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd84d2-fba9-43c1-ab23-e15ce1b3784e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:19.772195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b0c338-bded-46b4-be0b-5ddcdfec2c5d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:19.809185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25893648-258b-4788-98d3-8631dbd2c184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:19.809947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25893648-258b-4788-98d3-8631dbd2c184 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:19.810806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25893648-258b-4788-98d3-8631dbd2c184 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:19.811092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25893648-258b-4788-98d3-8631dbd2c184 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:19.811849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25893648-258b-4788-98d3-8631dbd2c184 None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:19.812158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25893648-258b-4788-98d3-8631dbd2c184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:19.813552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25893648-258b-4788-98d3-8631dbd2c184 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:19.823461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 74/144] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:19.829484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25893648-258b-4788-98d3-8631dbd2c184 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:19.853737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 71/145] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:34:19 2022] POST /identity/v3/users => generated 283 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:19.926154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:19.957662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25893648-258b-4788-98d3-8631dbd2c184 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:20.007441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84edd6-b744-42da-b15f-e09fa4723dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:20.008379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84edd6-b744-42da-b15f-e09fa4723dba None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:20.008661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84edd6-b744-42da-b15f-e09fa4723dba None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:20.008935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84edd6-b744-42da-b15f-e09fa4723dba None admin] PATH_INFO: `/v3/users/83ad07150d134e288ea59a065b959f79` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:20.010176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84edd6-b744-42da-b15f-e09fa4723dba None admin] RBAC: Authorizing `identity:get_user(user_id=83ad07150d134e288ea59a065b959f79)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:20.011639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84edd6-b744-42da-b15f-e09fa4723dba None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:20.013846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 75/146] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:19 2022] GET /identity/v3/users/83ad07150d134e288ea59a065b959f79 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:20.019527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd84d2-fba9-43c1-ab23-e15ce1b3784e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:20.078823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:20.153290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd84d2-fba9-43c1-ab23-e15ce1b3784e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:20.192188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80162d-374c-4a01-8313-83aef8bce40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:20.192780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80162d-374c-4a01-8313-83aef8bce40f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:20.193074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80162d-374c-4a01-8313-83aef8bce40f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:20.193356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80162d-374c-4a01-8313-83aef8bce40f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:20.267069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80162d-374c-4a01-8313-83aef8bce40f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:20.268593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80162d-374c-4a01-8313-83aef8bce40f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:20.317264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:20.389284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 72/147] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:20.394897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84edd6-b744-42da-b15f-e09fa4723dba None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:20.546311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84edd6-b744-42da-b15f-e09fa4723dba None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:20.586524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32917f-7cf0-47d9-a9cc-9ec1c10c5426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:20.587395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32917f-7cf0-47d9-a9cc-9ec1c10c5426 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:20.587724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32917f-7cf0-47d9-a9cc-9ec1c10c5426 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:20.588004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32917f-7cf0-47d9-a9cc-9ec1c10c5426 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:20.594580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d32917f-7cf0-47d9-a9cc-9ec1c10c5426 None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:20.596040 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d32917f-7cf0-47d9-a9cc-9ec1c10c5426 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:20.598061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 76/148] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:20 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:20.603950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80162d-374c-4a01-8313-83aef8bce40f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:20.738290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80162d-374c-4a01-8313-83aef8bce40f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:20.782751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25d416-059a-4d49-bf66-a6bb3edd59cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:20.783432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25d416-059a-4d49-bf66-a6bb3edd59cc None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:20.783718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25d416-059a-4d49-bf66-a6bb3edd59cc None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:20.783996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25d416-059a-4d49-bf66-a6bb3edd59cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:20.784841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25d416-059a-4d49-bf66-a6bb3edd59cc None admin] RBAC: Adding query filter params (role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=c89e3b28dc2a41c1822590c6a086be14, user.id=83ad07150d134e288ea59a065b959f79) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:20.785153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25d416-059a-4d49-bf66-a6bb3edd59cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:20.786655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25d416-059a-4d49-bf66-a6bb3edd59cc None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:20.795155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 73/149] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:20 2022] GET /identity/v3/role_assignments?user.id=83ad07150d134e288ea59a065b959f79&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:20.928776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d175e3f-28c1-4931-8b07-5a06dc8b7444 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:20.929085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d175e3f-28c1-4931-8b07-5a06dc8b7444 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:20.929264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d175e3f-28c1-4931-8b07-5a06dc8b7444 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:20.930961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 77/150] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:20.937035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba022c-70cd-4ba3-91be-93026749154f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:20.937676 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba022c-70cd-4ba3-91be-93026749154f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:20.938219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba022c-70cd-4ba3-91be-93026749154f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:20.955885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:20.969630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-53ba022c-70cd-4ba3-91be-93026749154f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:20.971272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-53ba022c-70cd-4ba3-91be-93026749154f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:21.066123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 74/151] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:21.092552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d175e3f-28c1-4931-8b07-5a06dc8b7444 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:21.109721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:21.211920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d175e3f-28c1-4931-8b07-5a06dc8b7444 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:21.259446 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97e309-18bf-4a12-9df2-07bc4ec85343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:21.260267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e309-18bf-4a12-9df2-07bc4ec85343 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:21.260557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e309-18bf-4a12-9df2-07bc4ec85343 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:21.260837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e309-18bf-4a12-9df2-07bc4ec85343 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:21.277793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97e309-18bf-4a12-9df2-07bc4ec85343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-bd97e309-18bf-4a12-9df2-07bc4ec85343 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:21.285973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:21.286865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:21.287641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 78/152] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:21.294770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba022c-70cd-4ba3-91be-93026749154f None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:21.419412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba022c-70cd-4ba3-91be-93026749154f None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:21.463559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543daffa-49eb-4d7e-8a52-cd080c583876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:21.464568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543daffa-49eb-4d7e-8a52-cd080c583876 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:21.465050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543daffa-49eb-4d7e-8a52-cd080c583876 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:21.465538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543daffa-49eb-4d7e-8a52-cd080c583876 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:21.467066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543daffa-49eb-4d7e-8a52-cd080c583876 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:21.467608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543daffa-49eb-4d7e-8a52-cd080c583876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:21.469291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543daffa-49eb-4d7e-8a52-cd080c583876 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:21.474201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:21.482619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 75/153] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:21.488392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97e309-18bf-4a12-9df2-07bc4ec85343 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:21.597409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:21.629926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97e309-18bf-4a12-9df2-07bc4ec85343 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:21.671802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d616b-d241-4af0-a268-718a9d6aa73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:21.672955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d616b-d241-4af0-a268-718a9d6aa73c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:21.673392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d616b-d241-4af0-a268-718a9d6aa73c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:21.673840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d616b-d241-4af0-a268-718a9d6aa73c None admin] PATH_INFO: `/v3/users/0b56f8efe53744f3ae9bc5ec439bbfcd` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:21.675223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d616b-d241-4af0-a268-718a9d6aa73c None admin] RBAC: Authorizing `identity:get_user(user_id=0b56f8efe53744f3ae9bc5ec439bbfcd)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:21.676817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d616b-d241-4af0-a268-718a9d6aa73c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:21.678639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 79/154] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:21 2022] GET /identity/v3/users/0b56f8efe53744f3ae9bc5ec439bbfcd => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:21.684456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543daffa-49eb-4d7e-8a52-cd080c583876 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:21.801660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543daffa-49eb-4d7e-8a52-cd080c583876 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:21.845130 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e24d2-4dfe-47bc-94b7-9e5350b1c26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:21.846419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e24d2-4dfe-47bc-94b7-9e5350b1c26f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:21.846867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e24d2-4dfe-47bc-94b7-9e5350b1c26f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:21.847290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e24d2-4dfe-47bc-94b7-9e5350b1c26f None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:21.849038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e24d2-4dfe-47bc-94b7-9e5350b1c26f None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:21.850705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e24d2-4dfe-47bc-94b7-9e5350b1c26f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:21.852904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 76/155] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:21 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:21.858466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d616b-d241-4af0-a268-718a9d6aa73c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:21.984185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d616b-d241-4af0-a268-718a9d6aa73c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:22.028315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd43a9e-6da9-4719-8773-dba6107e0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:22.029491 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd43a9e-6da9-4719-8773-dba6107e0933 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.034538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd43a9e-6da9-4719-8773-dba6107e0933 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.034538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd43a9e-6da9-4719-8773-dba6107e0933 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14/users/0b56f8efe53744f3ae9bc5ec439bbfcd/roles/eb50a4dbe6434ab0847618153c619d1a` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.034538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd43a9e-6da9-4719-8773-dba6107e0933 None admin] RBAC: Authorizing `identity:create_grant(project_id=c89e3b28dc2a41c1822590c6a086be14, user_id=0b56f8efe53744f3ae9bc5ec439bbfcd, role_id=eb50a4dbe6434ab0847618153c619d1a)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:22.034538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd43a9e-6da9-4719-8773-dba6107e0933 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:22.162324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 80/156] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:34:21 2022] PUT /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14/users/0b56f8efe53744f3ae9bc5ec439bbfcd/roles/eb50a4dbe6434ab0847618153c619d1a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:22.333564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fc9a2-935d-4910-8be2-5bdc64804898 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.334650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fc9a2-935d-4910-8be2-5bdc64804898 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.335153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fc9a2-935d-4910-8be2-5bdc64804898 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.336873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 77/157] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:22.343063 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a314adc-6823-4c1c-9438-50804e77c00a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.343653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a314adc-6823-4c1c-9438-50804e77c00a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.344136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a314adc-6823-4c1c-9438-50804e77c00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.349579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:22.366841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8a314adc-6823-4c1c-9438-50804e77c00a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:22.368138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8a314adc-6823-4c1c-9438-50804e77c00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:22.464392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 81/158] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:22.516449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fc9a2-935d-4910-8be2-5bdc64804898 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:22.599780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:22.642582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fc9a2-935d-4910-8be2-5bdc64804898 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:22.648063 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cb4fb-ec6f-4ead-a4e1-55a23fddc318 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.648541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cb4fb-ec6f-4ead-a4e1-55a23fddc318 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.648617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cb4fb-ec6f-4ead-a4e1-55a23fddc318 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.650245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 82/159] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:22.659195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930bccd-a3f9-4ce0-bc5e-e6b8cd84d3ef None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.659485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930bccd-a3f9-4ce0-bc5e-e6b8cd84d3ef None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.659766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930bccd-a3f9-4ce0-bc5e-e6b8cd84d3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.685722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f930bccd-a3f9-4ce0-bc5e-e6b8cd84d3ef None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:22.686959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f930bccd-a3f9-4ce0-bc5e-e6b8cd84d3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:22.723574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a0877-f636-4a56-944c-d7eca07a9d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:22.724301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a0877-f636-4a56-944c-d7eca07a9d80 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.724589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a0877-f636-4a56-944c-d7eca07a9d80 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.724870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a0877-f636-4a56-944c-d7eca07a9d80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.751928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a0877-f636-4a56-944c-d7eca07a9d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-d95a0877-f636-4a56-944c-d7eca07a9d80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:22.759743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:22.760678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:22.761466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 78/160] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:22.772289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2e30d-a3bd-4fc9-ba6f-f22f983aa948 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.772289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2e30d-a3bd-4fc9-ba6f-f22f983aa948 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.772289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2e30d-a3bd-4fc9-ba6f-f22f983aa948 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.772937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 79/161] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:22.773537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2e30d-a3bd-4fc9-ba6f-f22f983aa948 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:22.822623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 83/162] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:22.825053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51224bfa-65f5-46fa-b9bd-9cce70d55e57 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:22.826313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51224bfa-65f5-46fa-b9bd-9cce70d55e57 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:22.828095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51224bfa-65f5-46fa-b9bd-9cce70d55e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:22.870922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-51224bfa-65f5-46fa-b9bd-9cce70d55e57 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:22.872142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-51224bfa-65f5-46fa-b9bd-9cce70d55e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:22.966183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:22.984255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 84/163] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:22.984865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51224bfa-65f5-46fa-b9bd-9cce70d55e57 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:23.082016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2e30d-a3bd-4fc9-ba6f-f22f983aa948 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:23.106110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51224bfa-65f5-46fa-b9bd-9cce70d55e57 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:23.144962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63309ae-c45b-4a51-ab70-2468279fd80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:23.145787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63309ae-c45b-4a51-ab70-2468279fd80e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:23.146107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63309ae-c45b-4a51-ab70-2468279fd80e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:23.146388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63309ae-c45b-4a51-ab70-2468279fd80e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:23.158558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63309ae-c45b-4a51-ab70-2468279fd80e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:23.162148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-f63309ae-c45b-4a51-ab70-2468279fd80e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:23.167946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:23.168819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:23.170278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 85/164] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:23.175008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63309ae-c45b-4a51-ab70-2468279fd80e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:23.188872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35109fc-f780-45de-851a-c7fd9f67ee43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:23.189683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35109fc-f780-45de-851a-c7fd9f67ee43 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:23.190000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35109fc-f780-45de-851a-c7fd9f67ee43 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:23.190286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35109fc-f780-45de-851a-c7fd9f67ee43 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:23.191050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35109fc-f780-45de-851a-c7fd9f67ee43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:23.191614 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35109fc-f780-45de-851a-c7fd9f67ee43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:23.193020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35109fc-f780-45de-851a-c7fd9f67ee43 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:23.223836 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 80/165] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:23.224718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35109fc-f780-45de-851a-c7fd9f67ee43 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:23.238557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:23.293094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63309ae-c45b-4a51-ab70-2468279fd80e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:23.332528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fefcf-2e62-4930-a872-14bcb1c92c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:23.333711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fefcf-2e62-4930-a872-14bcb1c92c09 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:23.334036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fefcf-2e62-4930-a872-14bcb1c92c09 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:23.334316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6fefcf-2e62-4930-a872-14bcb1c92c09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:23.346828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6fefcf-2e62-4930-a872-14bcb1c92c09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-ea6fefcf-2e62-4930-a872-14bcb1c92c09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:23.355100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:23.355999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:23.356784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:23.362164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 86/166] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:23.363087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fefcf-2e62-4930-a872-14bcb1c92c09 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:23.481034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6fefcf-2e62-4930-a872-14bcb1c92c09 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:23.483521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35109fc-f780-45de-851a-c7fd9f67ee43 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:23.496930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:23.521009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39af32-3cc0-435e-84a1-1342b6d8d7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:23.522529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39af32-3cc0-435e-84a1-1342b6d8d7b8 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:23.522965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39af32-3cc0-435e-84a1-1342b6d8d7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:23.523391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39af32-3cc0-435e-84a1-1342b6d8d7b8 None admin] PATH_INFO: `/v3/users/d2ffbc8434d34115805dcbc2a72dc671` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:23.524761 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39af32-3cc0-435e-84a1-1342b6d8d7b8 None admin] RBAC: Authorizing `identity:get_user(user_id=d2ffbc8434d34115805dcbc2a72dc671)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:23.526838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39af32-3cc0-435e-84a1-1342b6d8d7b8 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:23.529458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 87/167] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:23 2022] GET /identity/v3/users/d2ffbc8434d34115805dcbc2a72dc671 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:23.530667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39af32-3cc0-435e-84a1-1342b6d8d7b8 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:23.558155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15569de-05f3-441c-b230-3e773140a61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:23.559140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15569de-05f3-441c-b230-3e773140a61f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:23.559251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15569de-05f3-441c-b230-3e773140a61f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:23.559561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15569de-05f3-441c-b230-3e773140a61f None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:23.560461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15569de-05f3-441c-b230-3e773140a61f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:23.560855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15569de-05f3-441c-b230-3e773140a61f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:23.562295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15569de-05f3-441c-b230-3e773140a61f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:23.581502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 81/168] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:23.582217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15569de-05f3-441c-b230-3e773140a61f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:23.671971 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:23.715046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39af32-3cc0-435e-84a1-1342b6d8d7b8 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:23.726504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15569de-05f3-441c-b230-3e773140a61f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:23.770301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8561386-3ed1-4ce6-8342-63830de04357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:23.771204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e4135-960e-490d-a0df-b960143f8869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:23.771395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8561386-3ed1-4ce6-8342-63830de04357 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:23.771811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8561386-3ed1-4ce6-8342-63830de04357 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:23.772170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4135-960e-490d-a0df-b960143f8869 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:23.772366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8561386-3ed1-4ce6-8342-63830de04357 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:23.772641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4135-960e-490d-a0df-b960143f8869 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:23.773015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4135-960e-490d-a0df-b960143f8869 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:23.774967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e4135-960e-490d-a0df-b960143f8869 None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:23.774967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e4135-960e-490d-a0df-b960143f8869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:23.775775 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e4135-960e-490d-a0df-b960143f8869 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:23.792289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 88/169] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:23.793173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e4135-960e-490d-a0df-b960143f8869 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:23.848143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:23.885431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8561386-3ed1-4ce6-8342-63830de04357 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:23.887358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8561386-3ed1-4ce6-8342-63830de04357 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:23.904800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e4135-960e-490d-a0df-b960143f8869 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:23.944909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94954069-353b-48ba-a6db-d45602f1346a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:23.945838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94954069-353b-48ba-a6db-d45602f1346a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:23.946126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94954069-353b-48ba-a6db-d45602f1346a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:23.946405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94954069-353b-48ba-a6db-d45602f1346a None admin] PATH_INFO: `/v3/users/83ad07150d134e288ea59a065b959f79` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:23.947882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94954069-353b-48ba-a6db-d45602f1346a None admin] RBAC: Authorizing `identity:get_user(user_id=83ad07150d134e288ea59a065b959f79)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:23.949349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94954069-353b-48ba-a6db-d45602f1346a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:23.951060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 89/170] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:23 2022] GET /identity/v3/users/83ad07150d134e288ea59a065b959f79 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:23.958865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94954069-353b-48ba-a6db-d45602f1346a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:24.011615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:24.111236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 82/171] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:24.112187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8561386-3ed1-4ce6-8342-63830de04357 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:24.160143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94954069-353b-48ba-a6db-d45602f1346a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:24.182616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:24.230079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebd5a1-08f0-408b-a238-b7f31a1b60a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:24.230973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebd5a1-08f0-408b-a238-b7f31a1b60a4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:24.231258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebd5a1-08f0-408b-a238-b7f31a1b60a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:24.231536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebd5a1-08f0-408b-a238-b7f31a1b60a4 None admin] PATH_INFO: `/v3/users/83ad07150d134e288ea59a065b959f79` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:24.234007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebd5a1-08f0-408b-a238-b7f31a1b60a4 None admin] RBAC: Authorizing `identity:get_user(user_id=83ad07150d134e288ea59a065b959f79)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:24.235472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebd5a1-08f0-408b-a238-b7f31a1b60a4 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:24.244416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 90/172] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:23 2022] GET /identity/v3/users/83ad07150d134e288ea59a065b959f79 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:24.245306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebd5a1-08f0-408b-a238-b7f31a1b60a4 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:24.326270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8561386-3ed1-4ce6-8342-63830de04357 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:24.381621 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ef286-77a3-4ac6-9549-c46cd4a9c973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:24.383148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ef286-77a3-4ac6-9549-c46cd4a9c973 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:24.384046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ef286-77a3-4ac6-9549-c46cd4a9c973 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:24.384534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ef286-77a3-4ac6-9549-c46cd4a9c973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:24.407619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebd5a1-08f0-408b-a238-b7f31a1b60a4 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:24.411511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:24.446325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb45874-9fe4-4704-9480-e6786521af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:24.447296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb45874-9fe4-4704-9480-e6786521af8c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:24.447593 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb45874-9fe4-4704-9480-e6786521af8c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:24.447872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb45874-9fe4-4704-9480-e6786521af8c None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:24.449107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb45874-9fe4-4704-9480-e6786521af8c None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:24.450602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb45874-9fe4-4704-9480-e6786521af8c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:24.457271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 91/173] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:24 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:24.457271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75990b-b3e1-4c35-8b16-c5e5cdb5d886 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:24.457271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75990b-b3e1-4c35-8b16-c5e5cdb5d886 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:24.457271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75990b-b3e1-4c35-8b16-c5e5cdb5d886 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:24.457271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 92/174] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:24.457271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75990b-b3e1-4c35-8b16-c5e5cdb5d886 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:24.483293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ef286-77a3-4ac6-9549-c46cd4a9c973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:24.484792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ef286-77a3-4ac6-9549-c46cd4a9c973 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:24.520659 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:24.596375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75990b-b3e1-4c35-8b16-c5e5cdb5d886 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:24.601431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 83/175] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:24.602918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41589dc-bf06-45cb-82ad-b5faf89e8e50 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:24.603215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41589dc-bf06-45cb-82ad-b5faf89e8e50 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:24.603499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41589dc-bf06-45cb-82ad-b5faf89e8e50 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:24.604856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 84/176] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:24.609081 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41589dc-bf06-45cb-82ad-b5faf89e8e50 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:24.647598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b03f95-62ca-4f3d-9cfa-e7b45b192c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:24.648258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b03f95-62ca-4f3d-9cfa-e7b45b192c98 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:24.648539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b03f95-62ca-4f3d-9cfa-e7b45b192c98 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:24.648814 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b03f95-62ca-4f3d-9cfa-e7b45b192c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:24.666038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:24.719602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41589dc-bf06-45cb-82ad-b5faf89e8e50 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:24.719968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b03f95-62ca-4f3d-9cfa-e7b45b192c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:24.721411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b03f95-62ca-4f3d-9cfa-e7b45b192c98 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:24.755459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915eab4-4053-45d6-8906-12949a290568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:24.756124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915eab4-4053-45d6-8906-12949a290568 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:24.756405 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915eab4-4053-45d6-8906-12949a290568 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:24.756676 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915eab4-4053-45d6-8906-12949a290568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:24.757514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915eab4-4053-45d6-8906-12949a290568 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=c89e3b28dc2a41c1822590c6a086be14, user.id=d2ffbc8434d34115805dcbc2a72dc671) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:24.757845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915eab4-4053-45d6-8906-12949a290568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:24.759290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915eab4-4053-45d6-8906-12949a290568 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:24.766911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 85/177] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:24 2022] GET /identity/v3/role_assignments?user.id=d2ffbc8434d34115805dcbc2a72dc671&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=abfb60dccca7470295258595a66a5d17 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:24.768170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceee77-aa8f-4adf-ab55-08134c19357b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:24.768468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceee77-aa8f-4adf-ab55-08134c19357b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:24.768757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceee77-aa8f-4adf-ab55-08134c19357b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:24.778373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:24.790828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-4fceee77-aa8f-4adf-ab55-08134c19357b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:24.796392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4fceee77-aa8f-4adf-ab55-08134c19357b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:24.858847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 93/178] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:24.860169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b03f95-62ca-4f3d-9cfa-e7b45b192c98 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:24.894374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 86/179] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:24.896495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ead7-b3fe-455e-9ddc-5a435604f050 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:24.896863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ead7-b3fe-455e-9ddc-5a435604f050 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:24.897217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ead7-b3fe-455e-9ddc-5a435604f050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:24.923910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-aad4ead7-b3fe-455e-9ddc-5a435604f050 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:24.924427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-aad4ead7-b3fe-455e-9ddc-5a435604f050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:24.983058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:24.988195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b03f95-62ca-4f3d-9cfa-e7b45b192c98 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:25.020385 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 87/180] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:25.022148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03050b6-1aeb-4bd3-a3aa-d8485a1b3a00 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.022533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03050b6-1aeb-4bd3-a3aa-d8485a1b3a00 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.022820 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03050b6-1aeb-4bd3-a3aa-d8485a1b3a00 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.024124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 88/181] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:25.028141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03050b6-1aeb-4bd3-a3aa-d8485a1b3a00 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.032451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e277a-8025-4a6a-8019-f6c77325cc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:25.033554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e277a-8025-4a6a-8019-f6c77325cc65 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.034045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e277a-8025-4a6a-8019-f6c77325cc65 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.034674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e277a-8025-4a6a-8019-f6c77325cc65 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.040249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e277a-8025-4a6a-8019-f6c77325cc65 None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:25.042362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e277a-8025-4a6a-8019-f6c77325cc65 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:25.044560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 94/182] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:24 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:25.045653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e277a-8025-4a6a-8019-f6c77325cc65 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.067691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:25.147413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03050b6-1aeb-4bd3-a3aa-d8485a1b3a00 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:25.168066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e277a-8025-4a6a-8019-f6c77325cc65 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:25.183734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33c98f-8032-41a4-ac93-35f78174ecae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:25.184605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33c98f-8032-41a4-ac93-35f78174ecae None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.184887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33c98f-8032-41a4-ac93-35f78174ecae None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.185699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33c98f-8032-41a4-ac93-35f78174ecae None admin] PATH_INFO: `/v3/projects/7a13aa03cdbd4db0871947b1fb2e05cf` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.186939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33c98f-8032-41a4-ac93-35f78174ecae None admin] RBAC: Authorizing `identity:get_project(project_id=7a13aa03cdbd4db0871947b1fb2e05cf)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:25.188421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33c98f-8032-41a4-ac93-35f78174ecae None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:25.195690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 89/183] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:25 2022] GET /identity/v3/projects/7a13aa03cdbd4db0871947b1fb2e05cf => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:25.195690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a1302-e6ba-4a76-9284-6d15228b1861 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.195690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a1302-e6ba-4a76-9284-6d15228b1861 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.195690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a1302-e6ba-4a76-9284-6d15228b1861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.208734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-c40a1302-e6ba-4a76-9284-6d15228b1861 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:25.209940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c40a1302-e6ba-4a76-9284-6d15228b1861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:25.210157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed426fcd-3a39-4741-b7d4-7d37705cb824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:25.210906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed426fcd-3a39-4741-b7d4-7d37705cb824 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.211186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed426fcd-3a39-4741-b7d4-7d37705cb824 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.211455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed426fcd-3a39-4741-b7d4-7d37705cb824 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.223677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed426fcd-3a39-4741-b7d4-7d37705cb824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-ed426fcd-3a39-4741-b7d4-7d37705cb824 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:25.231137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:25.232074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:25.232907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 95/184] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:25.233570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed426fcd-3a39-4741-b7d4-7d37705cb824 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.253177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:25.284732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 90/185] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:25.285666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a1302-e6ba-4a76-9284-6d15228b1861 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.318332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:25.342231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed426fcd-3a39-4741-b7d4-7d37705cb824 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:25.376634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d28a1-9c2a-492e-9245-e9182ef924aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:25.377291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d28a1-9c2a-492e-9245-e9182ef924aa None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.377571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d28a1-9c2a-492e-9245-e9182ef924aa None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.377873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d28a1-9c2a-492e-9245-e9182ef924aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.378704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d28a1-9c2a-492e-9245-e9182ef924aa None admin] RBAC: Adding query filter params (role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=c89e3b28dc2a41c1822590c6a086be14, user.id=83ad07150d134e288ea59a065b959f79) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:25.379010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d28a1-9c2a-492e-9245-e9182ef924aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:25.380461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d28a1-9c2a-492e-9245-e9182ef924aa None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:25.384949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a1302-e6ba-4a76-9284-6d15228b1861 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:25.386147 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 96/186] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:25 2022] GET /identity/v3/role_assignments?user.id=83ad07150d134e288ea59a065b959f79&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:25.386914 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d28a1-9c2a-492e-9245-e9182ef924aa None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.437405 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c3c9c-ef0e-4b85-bfb0-647d7a43decf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:25.438057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c3c9c-ef0e-4b85-bfb0-647d7a43decf None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.438349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c3c9c-ef0e-4b85-bfb0-647d7a43decf None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.438621 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c3c9c-ef0e-4b85-bfb0-647d7a43decf None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.440061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c3c9c-ef0e-4b85-bfb0-647d7a43decf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:25.442344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c3c9c-ef0e-4b85-bfb0-647d7a43decf None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:25.444415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 91/187] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:25.444939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c3c9c-ef0e-4b85-bfb0-647d7a43decf None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.496683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:25.510039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d28a1-9c2a-492e-9245-e9182ef924aa None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:25.551517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43ae09-81d1-4c90-9e10-e654d1311369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:25.552263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43ae09-81d1-4c90-9e10-e654d1311369 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.552702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43ae09-81d1-4c90-9e10-e654d1311369 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.553700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43ae09-81d1-4c90-9e10-e654d1311369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.553810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43ae09-81d1-4c90-9e10-e654d1311369 None admin] RBAC: Adding query filter params (role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=7a13aa03cdbd4db0871947b1fb2e05cf, user.id=83ad07150d134e288ea59a065b959f79) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:25.554174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43ae09-81d1-4c90-9e10-e654d1311369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:25.555762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43ae09-81d1-4c90-9e10-e654d1311369 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:25.562895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 97/188] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:25 2022] GET /identity/v3/role_assignments?user.id=83ad07150d134e288ea59a065b959f79&scope.project.id=7a13aa03cdbd4db0871947b1fb2e05cf&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:25.563686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a28101-d5a7-4da9-a804-83091ace5379 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.563971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a28101-d5a7-4da9-a804-83091ace5379 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.564245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a28101-d5a7-4da9-a804-83091ace5379 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.565616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 98/189] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:25.566473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a28101-d5a7-4da9-a804-83091ace5379 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.569573 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:25.579105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c3c9c-ef0e-4b85-bfb0-647d7a43decf None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:25.632549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a751144-344a-44a0-889a-9a9b8a03af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:25.633247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a751144-344a-44a0-889a-9a9b8a03af20 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.633532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a751144-344a-44a0-889a-9a9b8a03af20 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.633851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a751144-344a-44a0-889a-9a9b8a03af20 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.635150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a751144-344a-44a0-889a-9a9b8a03af20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:25.635634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a751144-344a-44a0-889a-9a9b8a03af20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:25.637137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a751144-344a-44a0-889a-9a9b8a03af20 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:25.647555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 92/190] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:25.648203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a751144-344a-44a0-889a-9a9b8a03af20 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.697212 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a28101-d5a7-4da9-a804-83091ace5379 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:25.734744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668cdc2-a888-4236-bcdd-a45184e0e62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:25.735564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668cdc2-a888-4236-bcdd-a45184e0e62b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.735846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668cdc2-a888-4236-bcdd-a45184e0e62b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.736119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668cdc2-a888-4236-bcdd-a45184e0e62b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.752948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668cdc2-a888-4236-bcdd-a45184e0e62b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:25.760268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a751144-344a-44a0-889a-9a9b8a03af20 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-8668cdc2-a888-4236-bcdd-a45184e0e62b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:25.761099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:25.762150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:25.763408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:25.764158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:25.764238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 99/191] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:25.764238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dcc67-ada8-4bd3-a678-714b9d9614ff None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.764238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dcc67-ada8-4bd3-a678-714b9d9614ff None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.764238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dcc67-ada8-4bd3-a678-714b9d9614ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.787691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-fc2dcc67-ada8-4bd3-a678-714b9d9614ff None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:25.789208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fc2dcc67-ada8-4bd3-a678-714b9d9614ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:25.802565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f4a16-78a9-4ab2-b9e8-18b23692b391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:25.806377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f4a16-78a9-4ab2-b9e8-18b23692b391 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:25.806377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f4a16-78a9-4ab2-b9e8-18b23692b391 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:25.806377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f4a16-78a9-4ab2-b9e8-18b23692b391 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:25.806377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f4a16-78a9-4ab2-b9e8-18b23692b391 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:25.806377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f4a16-78a9-4ab2-b9e8-18b23692b391 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:25.848059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:25.858465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 100/192] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:25.859272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dcc67-ada8-4bd3-a678-714b9d9614ff None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.925844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 93/193] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:34:25 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:25.925844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f4a16-78a9-4ab2-b9e8-18b23692b391 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:25.960834 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dcc67-ada8-4bd3-a678-714b9d9614ff None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:26.001460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a443e-dbe3-4685-a8a0-1ebfaea84766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:26.002369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a443e-dbe3-4685-a8a0-1ebfaea84766 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:26.002655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a443e-dbe3-4685-a8a0-1ebfaea84766 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:26.002930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a443e-dbe3-4685-a8a0-1ebfaea84766 None admin] PATH_INFO: `/v3/users/8908e69c1ff5455f82656940e2347362` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:26.020265 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a443e-dbe3-4685-a8a0-1ebfaea84766 None admin] RBAC: Authorizing `identity:get_user(user_id=8908e69c1ff5455f82656940e2347362)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:26.023084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a443e-dbe3-4685-a8a0-1ebfaea84766 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:26.028107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 101/194] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:25 2022] GET /identity/v3/users/8908e69c1ff5455f82656940e2347362 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:26.028107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a443e-dbe3-4685-a8a0-1ebfaea84766 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:26.054359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f4a16-78a9-4ab2-b9e8-18b23692b391 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:26.094893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:26.096253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814b27d-3f76-4a9d-9f23-6dcde39bf62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:26.097328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814b27d-3f76-4a9d-9f23-6dcde39bf62c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:26.097328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814b27d-3f76-4a9d-9f23-6dcde39bf62c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:26.097502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814b27d-3f76-4a9d-9f23-6dcde39bf62c None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:26.098280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814b27d-3f76-4a9d-9f23-6dcde39bf62c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:26.099560 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814b27d-3f76-4a9d-9f23-6dcde39bf62c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:26.100205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814b27d-3f76-4a9d-9f23-6dcde39bf62c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:26.111681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 94/195] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:26.117341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814b27d-3f76-4a9d-9f23-6dcde39bf62c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:26.144000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:26.157153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a443e-dbe3-4685-a8a0-1ebfaea84766 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:26.201605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522285b-4cb5-49bd-8175-ae18e3ffd59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:26.202701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522285b-4cb5-49bd-8175-ae18e3ffd59a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:26.203122 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522285b-4cb5-49bd-8175-ae18e3ffd59a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:26.203532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522285b-4cb5-49bd-8175-ae18e3ffd59a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:26.220714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522285b-4cb5-49bd-8175-ae18e3ffd59a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-d522285b-4cb5-49bd-8175-ae18e3ffd59a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:26.228908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:26.229943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:26.230778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 102/196] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:26.231426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522285b-4cb5-49bd-8175-ae18e3ffd59a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:26.252600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814b27d-3f76-4a9d-9f23-6dcde39bf62c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:26.298873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c92d6e-3b8e-4ab7-9da0-c7d197811f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:26.299961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c92d6e-3b8e-4ab7-9da0-c7d197811f40 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:26.300393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c92d6e-3b8e-4ab7-9da0-c7d197811f40 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:26.300695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c92d6e-3b8e-4ab7-9da0-c7d197811f40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:26.390855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522285b-4cb5-49bd-8175-ae18e3ffd59a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:26.404787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:26.418200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c92d6e-3b8e-4ab7-9da0-c7d197811f40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:26.419840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c92d6e-3b8e-4ab7-9da0-c7d197811f40 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:26.434743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a808f-17ad-41b3-a8a7-0e80f6e02bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:26.436339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a808f-17ad-41b3-a8a7-0e80f6e02bcf None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:26.436623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a808f-17ad-41b3-a8a7-0e80f6e02bcf None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:26.437239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a808f-17ad-41b3-a8a7-0e80f6e02bcf None admin] PATH_INFO: `/v3/users/8908e69c1ff5455f82656940e2347362` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:26.438804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a808f-17ad-41b3-a8a7-0e80f6e02bcf None admin] RBAC: Authorizing `identity:get_user(user_id=8908e69c1ff5455f82656940e2347362)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:26.440422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a808f-17ad-41b3-a8a7-0e80f6e02bcf None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:26.443018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 103/197] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:26 2022] GET /identity/v3/users/8908e69c1ff5455f82656940e2347362 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:26.444758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a808f-17ad-41b3-a8a7-0e80f6e02bcf None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:26.448304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:26.543890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 95/198] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:26.544784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c92d6e-3b8e-4ab7-9da0-c7d197811f40 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:26.574581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a808f-17ad-41b3-a8a7-0e80f6e02bcf None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:26.618331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0458559-674d-4136-8f1c-ba0f75dce7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:26.619621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0458559-674d-4136-8f1c-ba0f75dce7cc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:26.620516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0458559-674d-4136-8f1c-ba0f75dce7cc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:26.621425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0458559-674d-4136-8f1c-ba0f75dce7cc None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:26.623231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0458559-674d-4136-8f1c-ba0f75dce7cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:26.624018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0458559-674d-4136-8f1c-ba0f75dce7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:26.626343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0458559-674d-4136-8f1c-ba0f75dce7cc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:26.643755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 104/199] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:26.644941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0458559-674d-4136-8f1c-ba0f75dce7cc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:26.675341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c92d6e-3b8e-4ab7-9da0-c7d197811f40 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:26.703736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:26.706819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:26.721172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ef08c-3414-443c-b028-8b00dfa21d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:26.722024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ef08c-3414-443c-b028-8b00dfa21d7e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:26.722650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ef08c-3414-443c-b028-8b00dfa21d7e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:26.724156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ef08c-3414-443c-b028-8b00dfa21d7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:26.777990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0458559-674d-4136-8f1c-ba0f75dce7cc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:26.806607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ef08c-3414-443c-b028-8b00dfa21d7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:26.808355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ef08c-3414-443c-b028-8b00dfa21d7e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:26.815389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eb886-78cd-4743-9b58-7110f8e13d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:26.816535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eb886-78cd-4743-9b58-7110f8e13d0e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:26.816814 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eb886-78cd-4743-9b58-7110f8e13d0e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:26.817094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eb886-78cd-4743-9b58-7110f8e13d0e None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:26.818337 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816eb886-78cd-4743-9b58-7110f8e13d0e None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:26.819793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816eb886-78cd-4743-9b58-7110f8e13d0e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:26.822031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 105/200] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:26 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:26.822473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eb886-78cd-4743-9b58-7110f8e13d0e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:26.918588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 96/201] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:26.920309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ef08c-3414-443c-b028-8b00dfa21d7e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:26.927933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eb886-78cd-4743-9b58-7110f8e13d0e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:26.937230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:26.943318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:26.963710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75e5be-65f8-4e0c-a9ad-99fe80a8d91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:26.975073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75e5be-65f8-4e0c-a9ad-99fe80a8d91b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:26.975073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75e5be-65f8-4e0c-a9ad-99fe80a8d91b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:26.975073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75e5be-65f8-4e0c-a9ad-99fe80a8d91b None admin] PATH_INFO: `/v3/users/0b56f8efe53744f3ae9bc5ec439bbfcd` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:26.975073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75e5be-65f8-4e0c-a9ad-99fe80a8d91b None admin] RBAC: Authorizing `identity:get_user(user_id=0b56f8efe53744f3ae9bc5ec439bbfcd)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:26.975073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75e5be-65f8-4e0c-a9ad-99fe80a8d91b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:26.975073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 106/202] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:26 2022] GET /identity/v3/users/0b56f8efe53744f3ae9bc5ec439bbfcd => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:26.975475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75e5be-65f8-4e0c-a9ad-99fe80a8d91b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:27.031494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ef08c-3414-443c-b028-8b00dfa21d7e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:27.074786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5a1d8-d6f4-4e26-8229-3e74ffa6c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:27.074786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5a1d8-d6f4-4e26-8229-3e74ffa6c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:27.074786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5a1d8-d6f4-4e26-8229-3e74ffa6c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:27.074786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5a1d8-d6f4-4e26-8229-3e74ffa6c0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:27.075154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5a1d8-d6f4-4e26-8229-3e74ffa6c0b8 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=488c0084475d48098c54c5f7ee0fe92f, user.id=8908e69c1ff5455f82656940e2347362) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:27.075154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5a1d8-d6f4-4e26-8229-3e74ffa6c0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:27.075154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5a1d8-d6f4-4e26-8229-3e74ffa6c0b8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:27.077302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75e5be-65f8-4e0c-a9ad-99fe80a8d91b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:27.081521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 97/203] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:26 2022] GET /identity/v3/role_assignments?user.id=8908e69c1ff5455f82656940e2347362&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=abfb60dccca7470295258595a66a5d17 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:27.082580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5a1d8-d6f4-4e26-8229-3e74ffa6c0b8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:27.115448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566225a2-bad8-4a7c-9d53-725387df3484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:27.116426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566225a2-bad8-4a7c-9d53-725387df3484 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:27.116767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566225a2-bad8-4a7c-9d53-725387df3484 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:27.117045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566225a2-bad8-4a7c-9d53-725387df3484 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:27.118348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566225a2-bad8-4a7c-9d53-725387df3484 None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:27.119926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566225a2-bad8-4a7c-9d53-725387df3484 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:27.122174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 107/204] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:26 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:27.131001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566225a2-bad8-4a7c-9d53-725387df3484 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:27.186260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:27.200239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5a1d8-d6f4-4e26-8229-3e74ffa6c0b8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:27.216963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:27.241165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c74a9-f9cc-4e51-a19c-4c9129ab02d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:27.242200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c74a9-f9cc-4e51-a19c-4c9129ab02d3 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:27.242678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c74a9-f9cc-4e51-a19c-4c9129ab02d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:27.243143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c74a9-f9cc-4e51-a19c-4c9129ab02d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:27.294667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566225a2-bad8-4a7c-9d53-725387df3484 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:27.323454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c74a9-f9cc-4e51-a19c-4c9129ab02d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:27.325361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c74a9-f9cc-4e51-a19c-4c9129ab02d3 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:27.335523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1916b4-88cb-4be1-b68e-78aeaba2f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:27.336437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1916b4-88cb-4be1-b68e-78aeaba2f452 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:27.336437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1916b4-88cb-4be1-b68e-78aeaba2f452 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:27.336737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1916b4-88cb-4be1-b68e-78aeaba2f452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:27.337576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1916b4-88cb-4be1-b68e-78aeaba2f452 None admin] RBAC: Adding query filter params (role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=488c0084475d48098c54c5f7ee0fe92f, user.id=8908e69c1ff5455f82656940e2347362) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:27.337913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1916b4-88cb-4be1-b68e-78aeaba2f452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:27.339419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1916b4-88cb-4be1-b68e-78aeaba2f452 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:27.345914 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 108/205] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:27 2022] GET /identity/v3/role_assignments?user.id=8908e69c1ff5455f82656940e2347362&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:27.423946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:27.442718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 98/206] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:27.448718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1916b4-88cb-4be1-b68e-78aeaba2f452 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:27.584501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1916b4-88cb-4be1-b68e-78aeaba2f452 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:27.595621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:27.620701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fb096-7790-4445-be3d-43bcf74c4abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:27.621576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fb096-7790-4445-be3d-43bcf74c4abe None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:27.621890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fb096-7790-4445-be3d-43bcf74c4abe None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:27.622170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fb096-7790-4445-be3d-43bcf74c4abe None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:27.623403 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fb096-7790-4445-be3d-43bcf74c4abe None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:27.624856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fb096-7790-4445-be3d-43bcf74c4abe None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:27.626984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 109/207] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:27 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:27.632721 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c74a9-f9cc-4e51-a19c-4c9129ab02d3 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:27.770657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c74a9-f9cc-4e51-a19c-4c9129ab02d3 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:27.818516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0e563-9b59-486f-8b6c-49151cc0d678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:27.819467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e563-9b59-486f-8b6c-49151cc0d678 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:27.819904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e563-9b59-486f-8b6c-49151cc0d678 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:27.820329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0e563-9b59-486f-8b6c-49151cc0d678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:27.821329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0e563-9b59-486f-8b6c-49151cc0d678 None admin] RBAC: Adding query filter params (role.id=eb50a4dbe6434ab0847618153c619d1a, scope.project.id=c89e3b28dc2a41c1822590c6a086be14, user.id=0b56f8efe53744f3ae9bc5ec439bbfcd) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:27.821802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0e563-9b59-486f-8b6c-49151cc0d678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:27.823570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee0e563-9b59-486f-8b6c-49151cc0d678 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:27.830132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 99/208] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:27 2022] GET /identity/v3/role_assignments?user.id=0b56f8efe53744f3ae9bc5ec439bbfcd&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=eb50a4dbe6434ab0847618153c619d1a => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:28.210941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e284e-0e32-491f-9b35-ef2cdad81896 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:28.211807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e284e-0e32-491f-9b35-ef2cdad81896 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:28.212239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e284e-0e32-491f-9b35-ef2cdad81896 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:28.213755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 110/209] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:28.219912 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c51143-1da9-4f10-8fde-0535f4f2f540 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:28.220454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c51143-1da9-4f10-8fde-0535f4f2f540 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:28.220928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c51143-1da9-4f10-8fde-0535f4f2f540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:28.239424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e3c51143-1da9-4f10-8fde-0535f4f2f540 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:28.240932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e3c51143-1da9-4f10-8fde-0535f4f2f540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:28.296248 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:28.323110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 100/210] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:28.349611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e284e-0e32-491f-9b35-ef2cdad81896 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:28.399853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a655-6a81-4888-8b18-bf532eb00d09 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:28.400499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a655-6a81-4888-8b18-bf532eb00d09 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:28.401023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217a655-6a81-4888-8b18-bf532eb00d09 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:28.402675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 101/211] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:28.413327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e98264-6246-42da-9c6c-ed28a42ddec5 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:28.413961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e98264-6246-42da-9c6c-ed28a42ddec5 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:28.414452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e98264-6246-42da-9c6c-ed28a42ddec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:28.436930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-34e98264-6246-42da-9c6c-ed28a42ddec5 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:28.438590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-34e98264-6246-42da-9c6c-ed28a42ddec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:28.478017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e284e-0e32-491f-9b35-ef2cdad81896 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:28.519781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08ed3a-c8f3-49cf-9ac4-73218ac498d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:28.520860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08ed3a-c8f3-49cf-9ac4-73218ac498d2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:28.521289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08ed3a-c8f3-49cf-9ac4-73218ac498d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:28.521708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08ed3a-c8f3-49cf-9ac4-73218ac498d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:28.524599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 102/212] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:28.533123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08ed3a-c8f3-49cf-9ac4-73218ac498d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-fe08ed3a-c8f3-49cf-9ac4-73218ac498d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:28.540853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:28.541963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:28.542968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 111/213] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:28.549086 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e98264-6246-42da-9c6c-ed28a42ddec5 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:28.554823 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08ed3a-c8f3-49cf-9ac4-73218ac498d2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:28.602221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:28.656294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e98264-6246-42da-9c6c-ed28a42ddec5 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:28.666778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:28.691385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08ed3a-c8f3-49cf-9ac4-73218ac498d2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:28.693256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458186a4-a196-454d-8fc4-0b79688f0057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:28.693964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458186a4-a196-454d-8fc4-0b79688f0057 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:28.694435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458186a4-a196-454d-8fc4-0b79688f0057 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:28.694711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458186a4-a196-454d-8fc4-0b79688f0057 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:28.695532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458186a4-a196-454d-8fc4-0b79688f0057 None admin] RBAC: Adding query filter params (name=member) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:28.696009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458186a4-a196-454d-8fc4-0b79688f0057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:28.697625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458186a4-a196-454d-8fc4-0b79688f0057 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:28.708245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 103/214] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:28.713668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458186a4-a196-454d-8fc4-0b79688f0057 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:28.735248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f2023-98cb-4f3e-8d42-bfb49b838f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:28.736330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f2023-98cb-4f3e-8d42-bfb49b838f78 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:28.736700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f2023-98cb-4f3e-8d42-bfb49b838f78 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:28.737035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f2023-98cb-4f3e-8d42-bfb49b838f78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:28.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f2023-98cb-4f3e-8d42-bfb49b838f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:28.740273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f2023-98cb-4f3e-8d42-bfb49b838f78 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:28.745891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 112/215] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:28.749304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f2023-98cb-4f3e-8d42-bfb49b838f78 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:28.847996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458186a4-a196-454d-8fc4-0b79688f0057 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:28.891905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434ba9e-2f92-4703-a332-b4b0414a0c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:28.893195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434ba9e-2f92-4703-a332-b4b0414a0c9e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:28.893682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434ba9e-2f92-4703-a332-b4b0414a0c9e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:28.894200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434ba9e-2f92-4703-a332-b4b0414a0c9e None admin] PATH_INFO: `/v3/users/2be2796155ec431496bdf07cdf8c4d34` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:28.894726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f2023-98cb-4f3e-8d42-bfb49b838f78 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:28.897230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:28.912818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434ba9e-2f92-4703-a332-b4b0414a0c9e None admin] RBAC: Authorizing `identity:get_user(user_id=2be2796155ec431496bdf07cdf8c4d34)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:28.914696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434ba9e-2f92-4703-a332-b4b0414a0c9e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:28.916719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 104/216] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:28 2022] GET /identity/v3/users/2be2796155ec431496bdf07cdf8c4d34 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:28.923188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434ba9e-2f92-4703-a332-b4b0414a0c9e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:28.942133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea273-fba5-4890-ad27-061a90bb87e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:28.943063 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea273-fba5-4890-ad27-061a90bb87e5 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:28.943640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea273-fba5-4890-ad27-061a90bb87e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:28.944092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea273-fba5-4890-ad27-061a90bb87e5 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:28.945459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ea273-fba5-4890-ad27-061a90bb87e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:28.947255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ea273-fba5-4890-ad27-061a90bb87e5 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:28.963050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:29.043833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434ba9e-2f92-4703-a332-b4b0414a0c9e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:29.055264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 113/217] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:34:28 2022] POST /identity/v3/users => generated 297 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:34:29.086544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7e464-9b07-4de9-918a-250448da2e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:29.087588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7e464-9b07-4de9-918a-250448da2e37 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:29.088636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7e464-9b07-4de9-918a-250448da2e37 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:29.089096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7e464-9b07-4de9-918a-250448da2e37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:29.168651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7e464-9b07-4de9-918a-250448da2e37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:29.171024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba7e464-9b07-4de9-918a-250448da2e37 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:29.205275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:29.283323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 105/218] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:29.289209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea273-fba5-4890-ad27-061a90bb87e5 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:29.425493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea273-fba5-4890-ad27-061a90bb87e5 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:29.472604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21689449-97cd-4f8f-9276-a4a6f0f46b63 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:29.473255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21689449-97cd-4f8f-9276-a4a6f0f46b63 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:29.473735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21689449-97cd-4f8f-9276-a4a6f0f46b63 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:29.475137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e63dc-bcbe-4a70-b579-c8cfe3646b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:29.475625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 106/219] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:29.479550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e63dc-bcbe-4a70-b579-c8cfe3646b2f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:29.479550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e63dc-bcbe-4a70-b579-c8cfe3646b2f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:29.479550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e63dc-bcbe-4a70-b579-c8cfe3646b2f None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:29.479550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e63dc-bcbe-4a70-b579-c8cfe3646b2f None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:29.480178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e63dc-bcbe-4a70-b579-c8cfe3646b2f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:29.481477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228fe46-3694-435c-b494-c07e67468a7d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:29.482227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 114/220] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:29 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:29.482585 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228fe46-3694-435c-b494-c07e67468a7d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:29.483068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228fe46-3694-435c-b494-c07e67468a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:29.488083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e63dc-bcbe-4a70-b579-c8cfe3646b2f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:29.501551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d228fe46-3694-435c-b494-c07e67468a7d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:29.503140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d228fe46-3694-435c-b494-c07e67468a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:29.524617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:29.580103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 107/221] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:29.605387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228fe46-3694-435c-b494-c07e67468a7d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:29.674210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:29.679880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e63dc-bcbe-4a70-b579-c8cfe3646b2f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:29.721775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228fe46-3694-435c-b494-c07e67468a7d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:29.760126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306253ee-02fe-4188-80a3-555e6e4b4c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:29.761291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306253ee-02fe-4188-80a3-555e6e4b4c06 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:29.761779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306253ee-02fe-4188-80a3-555e6e4b4c06 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:29.762313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306253ee-02fe-4188-80a3-555e6e4b4c06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:29.774343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306253ee-02fe-4188-80a3-555e6e4b4c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-306253ee-02fe-4188-80a3-555e6e4b4c06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:29.782993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:29.783888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:29.784690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:29.787109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 108/222] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:29.791956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f38cc-cdf2-4cd9-b1e3-4a471b96f8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:29.791956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f38cc-cdf2-4cd9-b1e3-4a471b96f8f7 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:29.791956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f38cc-cdf2-4cd9-b1e3-4a471b96f8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:29.791956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f38cc-cdf2-4cd9-b1e3-4a471b96f8f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:29.792264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f38cc-cdf2-4cd9-b1e3-4a471b96f8f7 None admin] RBAC: Adding query filter params (role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=488c0084475d48098c54c5f7ee0fe92f, user.id=2be2796155ec431496bdf07cdf8c4d34) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:29.792264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f38cc-cdf2-4cd9-b1e3-4a471b96f8f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:29.792908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f38cc-cdf2-4cd9-b1e3-4a471b96f8f7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:29.793653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306253ee-02fe-4188-80a3-555e6e4b4c06 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:29.810182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 115/223] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:29 2022] GET /identity/v3/role_assignments?user.id=2be2796155ec431496bdf07cdf8c4d34&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:29.866865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3f6d2-689e-4efd-88b6-fa1c1ac15d43 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:29.867211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3f6d2-689e-4efd-88b6-fa1c1ac15d43 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:29.867420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3f6d2-689e-4efd-88b6-fa1c1ac15d43 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:29.869034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 116/224] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:29.875344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73672f1a-d0f4-477d-82aa-c8f1965cab28 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:29.875638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73672f1a-d0f4-477d-82aa-c8f1965cab28 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:29.875911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73672f1a-d0f4-477d-82aa-c8f1965cab28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:29.910210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-73672f1a-d0f4-477d-82aa-c8f1965cab28 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:29.911522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-73672f1a-d0f4-477d-82aa-c8f1965cab28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:29.944451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306253ee-02fe-4188-80a3-555e6e4b4c06 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:29.975112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:30.003993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26720edc-a37a-440c-9816-ec3875265940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:30.004789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720edc-a37a-440c-9816-ec3875265940 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:30.005079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720edc-a37a-440c-9816-ec3875265940 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:30.005358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720edc-a37a-440c-9816-ec3875265940 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:30.006182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26720edc-a37a-440c-9816-ec3875265940 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:30.006492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26720edc-a37a-440c-9816-ec3875265940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:30.007910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26720edc-a37a-440c-9816-ec3875265940 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:30.019023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 109/225] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:30.024671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26720edc-a37a-440c-9816-ec3875265940 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:30.025625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 117/226] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:30.062128 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73672f1a-d0f4-477d-82aa-c8f1965cab28 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:30.110229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:30.221763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26720edc-a37a-440c-9816-ec3875265940 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:30.246142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73672f1a-d0f4-477d-82aa-c8f1965cab28 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:30.279791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57b8c4-1099-4b57-a62a-ce0d999a6533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:30.280670 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57b8c4-1099-4b57-a62a-ce0d999a6533 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:30.280954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57b8c4-1099-4b57-a62a-ce0d999a6533 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:30.281227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57b8c4-1099-4b57-a62a-ce0d999a6533 None admin] PATH_INFO: `/v3/users/8908e69c1ff5455f82656940e2347362` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:30.282518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57b8c4-1099-4b57-a62a-ce0d999a6533 None admin] RBAC: Authorizing `identity:get_user(user_id=8908e69c1ff5455f82656940e2347362)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:30.284897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57b8c4-1099-4b57-a62a-ce0d999a6533 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:30.286555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 110/227] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:30 2022] GET /identity/v3/users/8908e69c1ff5455f82656940e2347362 => generated 283 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:30.292958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57b8c4-1099-4b57-a62a-ce0d999a6533 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:30.314841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4f243-15d1-4e7f-a640-989677ca13e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:30.315665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f243-15d1-4e7f-a640-989677ca13e7 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:30.315943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f243-15d1-4e7f-a640-989677ca13e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:30.316469 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f243-15d1-4e7f-a640-989677ca13e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:30.336463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4f243-15d1-4e7f-a640-989677ca13e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-cae4f243-15d1-4e7f-a640-989677ca13e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:30.350969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:30.351851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:30.352638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:30.353935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 118/228] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:30.359869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4f243-15d1-4e7f-a640-989677ca13e7 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:30.389845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:30.474415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57b8c4-1099-4b57-a62a-ce0d999a6533 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:30.495281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:30.518097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4f243-15d1-4e7f-a640-989677ca13e7 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:30.555709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ed55e-2226-4e13-83b5-65932da01f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:30.557053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ed55e-2226-4e13-83b5-65932da01f58 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:30.558126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ed55e-2226-4e13-83b5-65932da01f58 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:30.558709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ed55e-2226-4e13-83b5-65932da01f58 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:30.562283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ed55e-2226-4e13-83b5-65932da01f58 None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:30.564102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ed55e-2226-4e13-83b5-65932da01f58 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:30.567041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 111/229] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:30 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:30.573288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ed55e-2226-4e13-83b5-65932da01f58 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:30.578718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091806f-e0c6-44f8-a2fb-fa8436508b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:30.579575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091806f-e0c6-44f8-a2fb-fa8436508b7e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:30.580069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091806f-e0c6-44f8-a2fb-fa8436508b7e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:30.580523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091806f-e0c6-44f8-a2fb-fa8436508b7e None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:30.581427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091806f-e0c6-44f8-a2fb-fa8436508b7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:30.581935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091806f-e0c6-44f8-a2fb-fa8436508b7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:30.584513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091806f-e0c6-44f8-a2fb-fa8436508b7e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:30.600338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 119/230] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:30.606737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091806f-e0c6-44f8-a2fb-fa8436508b7e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:30.779517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091806f-e0c6-44f8-a2fb-fa8436508b7e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:30.800114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ed55e-2226-4e13-83b5-65932da01f58 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:30.835986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:30.850290 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b2a85-eead-4594-8a4b-38900e07ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:30.854844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b2a85-eead-4594-8a4b-38900e07ca2f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:30.854844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b2a85-eead-4594-8a4b-38900e07ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:30.854844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b2a85-eead-4594-8a4b-38900e07ca2f None admin] PATH_INFO: `/v3/users/8908e69c1ff5455f82656940e2347362` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:30.854844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b2a85-eead-4594-8a4b-38900e07ca2f None admin] RBAC: Authorizing `identity:get_user(user_id=8908e69c1ff5455f82656940e2347362)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:30.855652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b2a85-eead-4594-8a4b-38900e07ca2f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:30.861973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 120/231] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:30 2022] GET /identity/v3/users/8908e69c1ff5455f82656940e2347362 => generated 283 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:30.866725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b2a85-eead-4594-8a4b-38900e07ca2f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:30.871370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38190e71-8516-4246-96d9-5ab2726d6a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:30.872580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38190e71-8516-4246-96d9-5ab2726d6a70 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:30.873067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38190e71-8516-4246-96d9-5ab2726d6a70 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:30.873561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38190e71-8516-4246-96d9-5ab2726d6a70 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f/users/8908e69c1ff5455f82656940e2347362/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:30.876926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38190e71-8516-4246-96d9-5ab2726d6a70 None admin] RBAC: Authorizing `identity:create_grant(project_id=488c0084475d48098c54c5f7ee0fe92f, user_id=8908e69c1ff5455f82656940e2347362, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:30.878661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38190e71-8516-4246-96d9-5ab2726d6a70 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:30.894022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:30.944754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 112/232] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:34:30 2022] PUT /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f/users/8908e69c1ff5455f82656940e2347362/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:31.011741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b2a85-eead-4594-8a4b-38900e07ca2f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:31.053947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd99c8-062e-4667-8dca-7ff41e83fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:31.054841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd99c8-062e-4667-8dca-7ff41e83fac6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.055126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd99c8-062e-4667-8dca-7ff41e83fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.055402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd99c8-062e-4667-8dca-7ff41e83fac6 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.056668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd99c8-062e-4667-8dca-7ff41e83fac6 None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:31.058345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd99c8-062e-4667-8dca-7ff41e83fac6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:31.060798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 121/233] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:30 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:31.067061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38190e71-8516-4246-96d9-5ab2726d6a70 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:31.190384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38190e71-8516-4246-96d9-5ab2726d6a70 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:31.230281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95494f70-9fe0-4338-a859-875ba4d9a3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:31.231258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95494f70-9fe0-4338-a859-875ba4d9a3cd None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.231546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95494f70-9fe0-4338-a859-875ba4d9a3cd None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.231891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95494f70-9fe0-4338-a859-875ba4d9a3cd None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f/users/8908e69c1ff5455f82656940e2347362/roles/2806b70c57be44f9b260ff5322bf02fb` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.234264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95494f70-9fe0-4338-a859-875ba4d9a3cd None admin] RBAC: Authorizing `identity:create_grant(project_id=488c0084475d48098c54c5f7ee0fe92f, user_id=8908e69c1ff5455f82656940e2347362, role_id=2806b70c57be44f9b260ff5322bf02fb)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:31.235815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95494f70-9fe0-4338-a859-875ba4d9a3cd None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:31.351082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 113/234] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:34:31 2022] PUT /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f/users/8908e69c1ff5455f82656940e2347362/roles/2806b70c57be44f9b260ff5322bf02fb => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:31.548179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1910f6f-13b1-4b4b-834a-765b34f8f5b8 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.548179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1910f6f-13b1-4b4b-834a-765b34f8f5b8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.548179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1910f6f-13b1-4b4b-834a-765b34f8f5b8 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.548179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 122/235] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:31.554845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa2d6-117e-437e-9cd3-4af5339d3da2 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.555467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa2d6-117e-437e-9cd3-4af5339d3da2 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.555971 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa2d6-117e-437e-9cd3-4af5339d3da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.574796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-234fa2d6-117e-437e-9cd3-4af5339d3da2 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:31.576948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-234fa2d6-117e-437e-9cd3-4af5339d3da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:31.609956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:31.650388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 114/236] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:31.674669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1910f6f-13b1-4b4b-834a-765b34f8f5b8 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:31.778023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:31.800614 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc41ddf-c41f-4d43-aad3-90701c8a1c0d None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.801116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc41ddf-c41f-4d43-aad3-90701c8a1c0d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.801190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc41ddf-c41f-4d43-aad3-90701c8a1c0d None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.802759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 115/237] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:34:31.808916 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1910f6f-13b1-4b4b-834a-765b34f8f5b8 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:31.814497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a5e7-08c8-436b-aeb6-37c7071dcb44 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.815140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a5e7-08c8-436b-aeb6-37c7071dcb44 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.815637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a5e7-08c8-436b-aeb6-37c7071dcb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.843399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f5a4a5e7-08c8-436b-aeb6-37c7071dcb44 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:31.846414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f5a4a5e7-08c8-436b-aeb6-37c7071dcb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:31.864420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a93192-569e-408b-8fba-a46842320a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:31.865159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a93192-569e-408b-8fba-a46842320a99 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:31.865444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a93192-569e-408b-8fba-a46842320a99 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:31.865723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a93192-569e-408b-8fba-a46842320a99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:31.867370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a93192-569e-408b-8fba-a46842320a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:31.869016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a93192-569e-408b-8fba-a46842320a99 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:31.871482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 123/238] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:31.877986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a93192-569e-408b-8fba-a46842320a99 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:31.978989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 116/239] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:32.031960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a93192-569e-408b-8fba-a46842320a99 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:32.045596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4a5e7-08c8-436b-aeb6-37c7071dcb44 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:32.100996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4fc2e-e48a-449a-b556-255592f64005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:32.101683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4fc2e-e48a-449a-b556-255592f64005 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.101992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4fc2e-e48a-449a-b556-255592f64005 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.102600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4fc2e-e48a-449a-b556-255592f64005 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.103577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4fc2e-e48a-449a-b556-255592f64005 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:32.105259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4fc2e-e48a-449a-b556-255592f64005 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:32.174572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:32.181190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4a5e7-08c8-436b-aeb6-37c7071dcb44 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:32.209166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 124/240] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:34:31 2022] POST /identity/v3/users => generated 293 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:32.211015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e250a9-8706-4359-ac65-3ae692bb8be2 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.211946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e250a9-8706-4359-ac65-3ae692bb8be2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.212236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e250a9-8706-4359-ac65-3ae692bb8be2 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.214431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 125/241] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:32.220532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df451a-007b-457c-b331-839ad56eda7f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.220881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df451a-007b-457c-b331-839ad56eda7f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.221094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df451a-007b-457c-b331-839ad56eda7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.227436 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26163135-65b9-4046-b328-2cdb17bb1eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:32.228686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26163135-65b9-4046-b328-2cdb17bb1eba None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.229251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26163135-65b9-4046-b328-2cdb17bb1eba None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.229856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26163135-65b9-4046-b328-2cdb17bb1eba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.240813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:32.244778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26163135-65b9-4046-b328-2cdb17bb1eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-26163135-65b9-4046-b328-2cdb17bb1eba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:32.254084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:32.258597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:32.259398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 117/242] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:32.262013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b1df451a-007b-457c-b331-839ad56eda7f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:32.262532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26163135-65b9-4046-b328-2cdb17bb1eba None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:32.263259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b1df451a-007b-457c-b331-839ad56eda7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:32.383964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26163135-65b9-4046-b328-2cdb17bb1eba None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:32.426569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7a36d-6007-444a-9e4f-8c67a30d69c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:32.427566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7a36d-6007-444a-9e4f-8c67a30d69c6 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.428002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7a36d-6007-444a-9e4f-8c67a30d69c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.429113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7a36d-6007-444a-9e4f-8c67a30d69c6 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.430292 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7a36d-6007-444a-9e4f-8c67a30d69c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:32.430800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7a36d-6007-444a-9e4f-8c67a30d69c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:32.432577 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7a36d-6007-444a-9e4f-8c67a30d69c6 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:32.446006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 118/243] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:32.451784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7a36d-6007-444a-9e4f-8c67a30d69c6 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:32.457209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 126/244] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:32.473671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:32.481781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df451a-007b-457c-b331-839ad56eda7f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:32.579438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7a36d-6007-444a-9e4f-8c67a30d69c6 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:32.602547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df451a-007b-457c-b331-839ad56eda7f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:32.631725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7e08a-9d81-49c8-a81d-11b5b85d690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:32.633604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7e08a-9d81-49c8-a81d-11b5b85d690d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.635478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7e08a-9d81-49c8-a81d-11b5b85d690d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.635913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7e08a-9d81-49c8-a81d-11b5b85d690d None admin] PATH_INFO: `/v3/users/6ae9b676cbb648f083e30ca48740d104` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.652182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7e08a-9d81-49c8-a81d-11b5b85d690d None admin] RBAC: Authorizing `identity:get_user(user_id=6ae9b676cbb648f083e30ca48740d104)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:32.654440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7e08a-9d81-49c8-a81d-11b5b85d690d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:32.658377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03e524-84e8-449d-96c1-d6928edfd48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:32.658377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03e524-84e8-449d-96c1-d6928edfd48e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.658663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03e524-84e8-449d-96c1-d6928edfd48e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.658793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 119/245] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:32 2022] GET /identity/v3/users/6ae9b676cbb648f083e30ca48740d104 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:32.658931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03e524-84e8-449d-96c1-d6928edfd48e None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.666136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:32.666955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7e08a-9d81-49c8-a81d-11b5b85d690d None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:32.672476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03e524-84e8-449d-96c1-d6928edfd48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-1b03e524-84e8-449d-96c1-d6928edfd48e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:32.682520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:32.683434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:32.684326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 127/246] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:32.688864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03e524-84e8-449d-96c1-d6928edfd48e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:32.769381 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:32.799257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7e08a-9d81-49c8-a81d-11b5b85d690d None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:32.841408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b60290-57d5-4ca2-858b-f4a87e606d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:32.842103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b60290-57d5-4ca2-858b-f4a87e606d2b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.842413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b60290-57d5-4ca2-858b-f4a87e606d2b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.842883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b60290-57d5-4ca2-858b-f4a87e606d2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.845883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03e524-84e8-449d-96c1-d6928edfd48e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:32.883379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090298e-058d-4819-b37b-6030470f05ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:32.884361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090298e-058d-4819-b37b-6030470f05ae None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:32.884818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090298e-058d-4819-b37b-6030470f05ae None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:32.885295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090298e-058d-4819-b37b-6030470f05ae None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:32.886225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090298e-058d-4819-b37b-6030470f05ae None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:32.886671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090298e-058d-4819-b37b-6030470f05ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:32.888777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090298e-058d-4819-b37b-6030470f05ae None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:32.898918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 128/247] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:32.904805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090298e-058d-4819-b37b-6030470f05ae None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:32.943658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b60290-57d5-4ca2-858b-f4a87e606d2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:32.946150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b60290-57d5-4ca2-858b-f4a87e606d2b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:33.031068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:33.112108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090298e-058d-4819-b37b-6030470f05ae None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:33.179348 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:33.221927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 120/248] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:33.222978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b60290-57d5-4ca2-858b-f4a87e606d2b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:33.239103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4b99e-73df-43a1-ac3c-4fde86f991a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:33.240008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4b99e-73df-43a1-ac3c-4fde86f991a3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:33.240293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4b99e-73df-43a1-ac3c-4fde86f991a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:33.240574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4b99e-73df-43a1-ac3c-4fde86f991a3 None admin] PATH_INFO: `/v3/users/2be2796155ec431496bdf07cdf8c4d34` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:33.246658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4b99e-73df-43a1-ac3c-4fde86f991a3 None admin] RBAC: Authorizing `identity:get_user(user_id=2be2796155ec431496bdf07cdf8c4d34)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:33.248672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4b99e-73df-43a1-ac3c-4fde86f991a3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:33.250742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 129/249] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:32 2022] GET /identity/v3/users/2be2796155ec431496bdf07cdf8c4d34 => generated 297 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:33.269886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4b99e-73df-43a1-ac3c-4fde86f991a3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:33.353476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b60290-57d5-4ca2-858b-f4a87e606d2b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:33.442576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d9476-70cf-41fd-865f-1b88f7e694ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:33.443467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d9476-70cf-41fd-865f-1b88f7e694ba None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:33.443753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d9476-70cf-41fd-865f-1b88f7e694ba None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:33.444043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d9476-70cf-41fd-865f-1b88f7e694ba None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:33.445308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d9476-70cf-41fd-865f-1b88f7e694ba None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:33.446813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d9476-70cf-41fd-865f-1b88f7e694ba None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:33.448700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 121/250] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:33 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:33.456918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d9476-70cf-41fd-865f-1b88f7e694ba None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:33.555301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4b99e-73df-43a1-ac3c-4fde86f991a3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:33.562181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:33.598221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:33.619010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1441b-1cb6-474a-b12e-90a3b3514e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:33.619891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1441b-1cb6-474a-b12e-90a3b3514e5b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:33.620179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1441b-1cb6-474a-b12e-90a3b3514e5b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:33.620465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1441b-1cb6-474a-b12e-90a3b3514e5b None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:33.623683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1441b-1cb6-474a-b12e-90a3b3514e5b None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:33.625152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1441b-1cb6-474a-b12e-90a3b3514e5b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:33.628923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 130/251] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:33 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:33.630225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16a778-a32f-4ca6-956e-53b3241d525d None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:33.630513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16a778-a32f-4ca6-956e-53b3241d525d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:33.630792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16a778-a32f-4ca6-956e-53b3241d525d None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:33.632065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 131/252] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:33.635257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16a778-a32f-4ca6-956e-53b3241d525d None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:33.659785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d9476-70cf-41fd-865f-1b88f7e694ba None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:33.723914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9c926-6962-499f-8514-aef2057873ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:33.724587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9c926-6962-499f-8514-aef2057873ab None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:33.724875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9c926-6962-499f-8514-aef2057873ab None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:33.725156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9c926-6962-499f-8514-aef2057873ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:33.726026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9c926-6962-499f-8514-aef2057873ab None admin] RBAC: Adding query filter params (role.id=eb50a4dbe6434ab0847618153c619d1a, scope.project.id=488c0084475d48098c54c5f7ee0fe92f, user.id=6ae9b676cbb648f083e30ca48740d104) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:33.726340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9c926-6962-499f-8514-aef2057873ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:33.727803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9c926-6962-499f-8514-aef2057873ab None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:33.738451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 122/253] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:33 2022] GET /identity/v3/role_assignments?user.id=6ae9b676cbb648f083e30ca48740d104&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=eb50a4dbe6434ab0847618153c619d1a => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:33.739702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f68c-2a95-4cee-bfe5-abcd8a0e5bf4 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:33.739991 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f68c-2a95-4cee-bfe5-abcd8a0e5bf4 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:33.740272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f68c-2a95-4cee-bfe5-abcd8a0e5bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:33.789090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ad86f68c-2a95-4cee-bfe5-abcd8a0e5bf4 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:33.790375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ad86f68c-2a95-4cee-bfe5-abcd8a0e5bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:33.837666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16a778-a32f-4ca6-956e-53b3241d525d None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:33.876776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a4b6d-9d22-4725-8d58-eaf8edcbdad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:33.878181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a4b6d-9d22-4725-8d58-eaf8edcbdad1 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:33.878707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a4b6d-9d22-4725-8d58-eaf8edcbdad1 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:33.879238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a4b6d-9d22-4725-8d58-eaf8edcbdad1 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f/users/2be2796155ec431496bdf07cdf8c4d34/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:33.881690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a4b6d-9d22-4725-8d58-eaf8edcbdad1 None admin] RBAC: Authorizing `identity:create_grant(project_id=488c0084475d48098c54c5f7ee0fe92f, user_id=2be2796155ec431496bdf07cdf8c4d34, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:33.883530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a4b6d-9d22-4725-8d58-eaf8edcbdad1 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:33.908830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 123/254] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:33.933375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86f68c-2a95-4cee-bfe5-abcd8a0e5bf4 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:33.974170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 132/255] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:34:33 2022] PUT /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f/users/2be2796155ec431496bdf07cdf8c4d34/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:33.987311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:34.078307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86f68c-2a95-4cee-bfe5-abcd8a0e5bf4 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:34.150067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaad75-a873-4725-b184-59e3d526e7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:34.151218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaad75-a873-4725-b184-59e3d526e7ca None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:34.151707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaad75-a873-4725-b184-59e3d526e7ca None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:34.152204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaad75-a873-4725-b184-59e3d526e7ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:34.170184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaad75-a873-4725-b184-59e3d526e7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-d2aaad75-a873-4725-b184-59e3d526e7ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:34.182543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:34.183511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:34.184321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 124/256] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:34.190231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a834ea-fa5c-4f1c-ae42-7bcf47be0513 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:34.190525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a834ea-fa5c-4f1c-ae42-7bcf47be0513 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:34.190811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a834ea-fa5c-4f1c-ae42-7bcf47be0513 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:34.192646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 133/257] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:34.193851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaad75-a873-4725-b184-59e3d526e7ca None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:34.197911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063ec4c-67a5-41fa-bb71-e2e22ec5a192 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:34.198188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063ec4c-67a5-41fa-bb71-e2e22ec5a192 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:34.198466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063ec4c-67a5-41fa-bb71-e2e22ec5a192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:34.234957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f063ec4c-67a5-41fa-bb71-e2e22ec5a192 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:34.236186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f063ec4c-67a5-41fa-bb71-e2e22ec5a192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:34.339940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:34.379354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 134/258] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:34.390178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:34.398107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaad75-a873-4725-b184-59e3d526e7ca None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:34.425144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063ec4c-67a5-41fa-bb71-e2e22ec5a192 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:34.470523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b50ca7-e85f-483e-a175-c4d25f950e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:34.471216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b50ca7-e85f-483e-a175-c4d25f950e1a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:34.471590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b50ca7-e85f-483e-a175-c4d25f950e1a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:34.471875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b50ca7-e85f-483e-a175-c4d25f950e1a None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:34.472653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b50ca7-e85f-483e-a175-c4d25f950e1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:34.472965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b50ca7-e85f-483e-a175-c4d25f950e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:34.476748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b50ca7-e85f-483e-a175-c4d25f950e1a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:34.494731 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 125/259] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:34.523462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b50ca7-e85f-483e-a175-c4d25f950e1a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:34.587245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063ec4c-67a5-41fa-bb71-e2e22ec5a192 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:34.623199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b65e6-73b6-4468-b921-cdeaed89df3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:34.624731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b65e6-73b6-4468-b921-cdeaed89df3a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:34.625164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b65e6-73b6-4468-b921-cdeaed89df3a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:34.625588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b65e6-73b6-4468-b921-cdeaed89df3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:34.636632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b65e6-73b6-4468-b921-cdeaed89df3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-af3b65e6-73b6-4468-b921-cdeaed89df3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:34.645106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:34.646243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:34.647317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:34.649543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 135/260] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:34.675710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b65e6-73b6-4468-b921-cdeaed89df3a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:34.687444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b50ca7-e85f-483e-a175-c4d25f950e1a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:34.739134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76b91-0878-46b9-8cda-ce1a74c5275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:34.740410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76b91-0878-46b9-8cda-ce1a74c5275f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:34.740914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76b91-0878-46b9-8cda-ce1a74c5275f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:34.741392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a76b91-0878-46b9-8cda-ce1a74c5275f None admin] PATH_INFO: `/v3/users/8908e69c1ff5455f82656940e2347362` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:34.746559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a76b91-0878-46b9-8cda-ce1a74c5275f None admin] RBAC: Authorizing `identity:get_user(user_id=8908e69c1ff5455f82656940e2347362)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:34.746707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:34.748780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a76b91-0878-46b9-8cda-ce1a74c5275f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:34.750895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 126/261] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:34 2022] GET /identity/v3/users/8908e69c1ff5455f82656940e2347362 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:34.770858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76b91-0878-46b9-8cda-ce1a74c5275f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:34.823304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b65e6-73b6-4468-b921-cdeaed89df3a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:34.834218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:34.910558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10ca65-cb4f-4463-91a9-c954a847a653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:34.911264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10ca65-cb4f-4463-91a9-c954a847a653 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:34.911550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10ca65-cb4f-4463-91a9-c954a847a653 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:34.911830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10ca65-cb4f-4463-91a9-c954a847a653 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:34.912595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10ca65-cb4f-4463-91a9-c954a847a653 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:34.912911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10ca65-cb4f-4463-91a9-c954a847a653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:34.914352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10ca65-cb4f-4463-91a9-c954a847a653 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:34.929343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 136/262] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:34.946862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10ca65-cb4f-4463-91a9-c954a847a653 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:34.956017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a76b91-0878-46b9-8cda-ce1a74c5275f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:35.023417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e158f15-2f99-46c5-b8a5-e464b54ed239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:35.024092 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158f15-2f99-46c5-b8a5-e464b54ed239 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:35.024377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158f15-2f99-46c5-b8a5-e464b54ed239 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:35.024656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158f15-2f99-46c5-b8a5-e464b54ed239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:35.181673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158f15-2f99-46c5-b8a5-e464b54ed239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:35.183535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158f15-2f99-46c5-b8a5-e464b54ed239 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:35.189557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10ca65-cb4f-4463-91a9-c954a847a653 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:35.192981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:35.234596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda403b-a5c0-44fa-aca7-872068c446cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:35.235784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda403b-a5c0-44fa-aca7-872068c446cc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:35.236292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda403b-a5c0-44fa-aca7-872068c446cc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:35.236734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda403b-a5c0-44fa-aca7-872068c446cc None admin] PATH_INFO: `/v3/users/8908e69c1ff5455f82656940e2347362` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:35.242555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda403b-a5c0-44fa-aca7-872068c446cc None admin] RBAC: Authorizing `identity:get_user(user_id=8908e69c1ff5455f82656940e2347362)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:35.244185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda403b-a5c0-44fa-aca7-872068c446cc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:35.245231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:35.246020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 137/263] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:34 2022] GET /identity/v3/users/8908e69c1ff5455f82656940e2347362 => generated 283 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:35.251596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda403b-a5c0-44fa-aca7-872068c446cc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:35.376171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 127/264] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:35.378013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf493ca-fa0c-43bf-b0bc-2848d0366116 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:35.378533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf493ca-fa0c-43bf-b0bc-2848d0366116 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:35.379016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf493ca-fa0c-43bf-b0bc-2848d0366116 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:35.380703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 128/265] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:35.381651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf493ca-fa0c-43bf-b0bc-2848d0366116 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:35.458739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda403b-a5c0-44fa-aca7-872068c446cc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:35.532565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039fdfa6-d8f6-4e67-b6fa-16dcc3a688f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:35.533592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fdfa6-d8f6-4e67-b6fa-16dcc3a688f3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:35.534139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fdfa6-d8f6-4e67-b6fa-16dcc3a688f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:35.534615 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039fdfa6-d8f6-4e67-b6fa-16dcc3a688f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:35.549237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf493ca-fa0c-43bf-b0bc-2848d0366116 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:35.585181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:35.592372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:35.595041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326b0b0-b796-4afc-abf7-edb87f5f49ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:35.601356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326b0b0-b796-4afc-abf7-edb87f5f49ad None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:35.601356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326b0b0-b796-4afc-abf7-edb87f5f49ad None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:35.601356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326b0b0-b796-4afc-abf7-edb87f5f49ad None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:35.601356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326b0b0-b796-4afc-abf7-edb87f5f49ad None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:35.601356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326b0b0-b796-4afc-abf7-edb87f5f49ad None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:35.603343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 129/266] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:35 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:35.604688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b1135-b4e1-4864-83d8-324be02e97c4 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:35.605593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b1135-b4e1-4864-83d8-324be02e97c4 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:35.606152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b1135-b4e1-4864-83d8-324be02e97c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:35.623326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039fdfa6-d8f6-4e67-b6fa-16dcc3a688f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:35.624945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039fdfa6-d8f6-4e67-b6fa-16dcc3a688f3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:35.628528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d88b1135-b4e1-4864-83d8-324be02e97c4 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:35.631042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d88b1135-b4e1-4864-83d8-324be02e97c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:35.715769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 130/267] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:35.717375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189f19c-3d4b-495a-b4f1-f11290072aca None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:35.717871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189f19c-3d4b-495a-b4f1-f11290072aca None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:35.718312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189f19c-3d4b-495a-b4f1-f11290072aca None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:35.719848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 131/268] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:35.720898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189f19c-3d4b-495a-b4f1-f11290072aca None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:35.736610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 138/269] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:35.738262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2973-8e10-410e-bffc-7204ebfa70d1 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:35.738755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2973-8e10-410e-bffc-7204ebfa70d1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:35.739234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2973-8e10-410e-bffc-7204ebfa70d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:35.756497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-6bbd2973-8e10-410e-bffc-7204ebfa70d1 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:35.758036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6bbd2973-8e10-410e-bffc-7204ebfa70d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:35.836535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189f19c-3d4b-495a-b4f1-f11290072aca None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:35.837771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 139/270] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:35.838939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2973-8e10-410e-bffc-7204ebfa70d1 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:35.842315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:35.866830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:35.877237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8a67e-415b-4567-97f3-6cf753eef35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:35.878581 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8a67e-415b-4567-97f3-6cf753eef35b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:35.879026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8a67e-415b-4567-97f3-6cf753eef35b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:35.879763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8a67e-415b-4567-97f3-6cf753eef35b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:35.880902 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8a67e-415b-4567-97f3-6cf753eef35b None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=488c0084475d48098c54c5f7ee0fe92f, user.id=8908e69c1ff5455f82656940e2347362) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:35.881360 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8a67e-415b-4567-97f3-6cf753eef35b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:35.887170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8a67e-415b-4567-97f3-6cf753eef35b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:35.894516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 132/271] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:35 2022] GET /identity/v3/role_assignments?user.id=8908e69c1ff5455f82656940e2347362&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=abfb60dccca7470295258595a66a5d17 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:35.895934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8a67e-415b-4567-97f3-6cf753eef35b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:35.965763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2973-8e10-410e-bffc-7204ebfa70d1 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.004206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d523387-6b87-49fc-b163-10292d286f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.005387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d523387-6b87-49fc-b163-10292d286f78 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.005847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d523387-6b87-49fc-b163-10292d286f78 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.006267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d523387-6b87-49fc-b163-10292d286f78 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.007672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d523387-6b87-49fc-b163-10292d286f78 None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:36.009357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d523387-6b87-49fc-b163-10292d286f78 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:36.011555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 140/272] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:35 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:36.012783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d523387-6b87-49fc-b163-10292d286f78 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:36.022517 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8a67e-415b-4567-97f3-6cf753eef35b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.063652 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912afed8-1098-482b-885e-ba0a32a3c16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.064657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912afed8-1098-482b-885e-ba0a32a3c16d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.065019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912afed8-1098-482b-885e-ba0a32a3c16d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.065367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912afed8-1098-482b-885e-ba0a32a3c16d None admin] PATH_INFO: `/v3/roles/member` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.094413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912afed8-1098-482b-885e-ba0a32a3c16d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-912afed8-1098-482b-885e-ba0a32a3c16d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:36.106905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:36.107851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:36.108655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:36.109833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:36.114241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 133/273] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:36.115146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912afed8-1098-482b-885e-ba0a32a3c16d None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:36.143890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d523387-6b87-49fc-b163-10292d286f78 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.176384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:36.183328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f66784-0b4f-4b9c-8382-5989fc2681bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.184442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f66784-0b4f-4b9c-8382-5989fc2681bb None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.185410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f66784-0b4f-4b9c-8382-5989fc2681bb None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.185877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f66784-0b4f-4b9c-8382-5989fc2681bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.187225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f66784-0b4f-4b9c-8382-5989fc2681bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:36.189272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f66784-0b4f-4b9c-8382-5989fc2681bb None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:36.190837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 141/274] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:36.192156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f66784-0b4f-4b9c-8382-5989fc2681bb None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:36.242822 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912afed8-1098-482b-885e-ba0a32a3c16d None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.288869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce1539-fb9f-4cd8-905f-72d6c68fd5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.289567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce1539-fb9f-4cd8-905f-72d6c68fd5be None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.290192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce1539-fb9f-4cd8-905f-72d6c68fd5be None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.290317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce1539-fb9f-4cd8-905f-72d6c68fd5be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.291226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce1539-fb9f-4cd8-905f-72d6c68fd5be None admin] RBAC: Adding query filter params (role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=488c0084475d48098c54c5f7ee0fe92f, user.id=8908e69c1ff5455f82656940e2347362) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:36.291608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce1539-fb9f-4cd8-905f-72d6c68fd5be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:36.293156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce1539-fb9f-4cd8-905f-72d6c68fd5be None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:36.301111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 134/275] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:36 2022] GET /identity/v3/role_assignments?user.id=8908e69c1ff5455f82656940e2347362&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:36.301805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce1539-fb9f-4cd8-905f-72d6c68fd5be None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:36.321227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f66784-0b4f-4b9c-8382-5989fc2681bb None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.372093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867803-9f03-4801-a525-57cab189d5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.373426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867803-9f03-4801-a525-57cab189d5e3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.373919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867803-9f03-4801-a525-57cab189d5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.374379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867803-9f03-4801-a525-57cab189d5e3 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.375995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3867803-9f03-4801-a525-57cab189d5e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:36.376503 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3867803-9f03-4801-a525-57cab189d5e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:36.378596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3867803-9f03-4801-a525-57cab189d5e3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:36.386121 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:36.391873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 142/276] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:36.397038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867803-9f03-4801-a525-57cab189d5e3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:36.434496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce1539-fb9f-4cd8-905f-72d6c68fd5be None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.452593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:36.487122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64151f-f868-4d39-82b2-85cec1c0b952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.488525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64151f-f868-4d39-82b2-85cec1c0b952 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.489222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64151f-f868-4d39-82b2-85cec1c0b952 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.491323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64151f-f868-4d39-82b2-85cec1c0b952 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.493165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64151f-f868-4d39-82b2-85cec1c0b952 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:36.494923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64151f-f868-4d39-82b2-85cec1c0b952 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:36.541059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867803-9f03-4801-a525-57cab189d5e3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.582800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7d3b9-8fcf-4dea-a91c-168fd76fe3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.583749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7d3b9-8fcf-4dea-a91c-168fd76fe3b6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.584155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7d3b9-8fcf-4dea-a91c-168fd76fe3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.584428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7d3b9-8fcf-4dea-a91c-168fd76fe3b6 None admin] PATH_INFO: `/v3/users/c6fbd651e2574e5dab86a987b2460310` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.587299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 135/277] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:34:36 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:36.588638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bab2f-e335-471c-a863-7e847703daa4 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.588928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bab2f-e335-471c-a863-7e847703daa4 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.589207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bab2f-e335-471c-a863-7e847703daa4 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.598944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 136/278] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:36.604306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7d3b9-8fcf-4dea-a91c-168fd76fe3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=c6fbd651e2574e5dab86a987b2460310)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:36.604826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca2798-caca-4496-9fe2-ec168b3d7f8c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.605116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca2798-caca-4496-9fe2-ec168b3d7f8c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.605394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca2798-caca-4496-9fe2-ec168b3d7f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.605910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7d3b9-8fcf-4dea-a91c-168fd76fe3b6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:36.608471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 143/279] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:36 2022] GET /identity/v3/users/c6fbd651e2574e5dab86a987b2460310 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:36.614395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7d3b9-8fcf-4dea-a91c-168fd76fe3b6 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:36.631908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-10ca2798-caca-4496-9fe2-ec168b3d7f8c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:36.633129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-10ca2798-caca-4496-9fe2-ec168b3d7f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:36.706182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:36.725528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 137/280] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:36.746971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7d3b9-8fcf-4dea-a91c-168fd76fe3b6 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.752773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca2798-caca-4496-9fe2-ec168b3d7f8c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:36.799269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7d1ec-3489-416e-bfa7-c943a645ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.799945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7d1ec-3489-416e-bfa7-c943a645ed7c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.800236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7d1ec-3489-416e-bfa7-c943a645ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.800516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7d1ec-3489-416e-bfa7-c943a645ed7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.801365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7d1ec-3489-416e-bfa7-c943a645ed7c None admin] RBAC: Adding query filter params (role.id=fda37aea043b4685b930f1d9bde2b0ec, user.id=c6fbd651e2574e5dab86a987b2460310, scope.system=all) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:36.801683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7d1ec-3489-416e-bfa7-c943a645ed7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:36.805082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7d1ec-3489-416e-bfa7-c943a645ed7c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:36.810420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 144/281] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jul 14 15:34:36 2022] GET /identity/v3/role_assignments?user.id=c6fbd651e2574e5dab86a987b2460310&scope.system=all&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:36.883298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:36.887183 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca2798-caca-4496-9fe2-ec168b3d7f8c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:36.928936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6eca-d5ee-4211-ae80-c8e16611b2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:36.929892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6eca-d5ee-4211-ae80-c8e16611b2dd None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:36.930253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6eca-d5ee-4211-ae80-c8e16611b2dd None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:36.930607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6eca-d5ee-4211-ae80-c8e16611b2dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:36.944376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6eca-d5ee-4211-ae80-c8e16611b2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-5e5d6eca-d5ee-4211-ae80-c8e16611b2dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:36.952797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:36.953954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:36.954898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:36.957436 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 138/282] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:36.962400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7d1ec-3489-416e-bfa7-c943a645ed7c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:37.039030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb93d7c-093a-4b33-9752-de88f5f359d5 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:37.039896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb93d7c-093a-4b33-9752-de88f5f359d5 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:37.039896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb93d7c-093a-4b33-9752-de88f5f359d5 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:37.043281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 139/283] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:34:37.049913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f3a3-be24-458d-89f0-dcfeed2b4840 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:37.049913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f3a3-be24-458d-89f0-dcfeed2b4840 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:37.049913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f3a3-be24-458d-89f0-dcfeed2b4840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:37.073842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-b824f3a3-be24-458d-89f0-dcfeed2b4840 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:37.074155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b824f3a3-be24-458d-89f0-dcfeed2b4840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:37.094461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7d1ec-3489-416e-bfa7-c943a645ed7c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:37.122664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:37.135176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e64f8-bf40-4a9f-adaa-8e74993612d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:37.136311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e64f8-bf40-4a9f-adaa-8e74993612d5 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:37.136788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e64f8-bf40-4a9f-adaa-8e74993612d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:37.137256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e64f8-bf40-4a9f-adaa-8e74993612d5 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:37.138262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e64f8-bf40-4a9f-adaa-8e74993612d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:37.138765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e64f8-bf40-4a9f-adaa-8e74993612d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:37.140528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e64f8-bf40-4a9f-adaa-8e74993612d5 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:37.149975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 145/284] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:37.156697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e64f8-bf40-4a9f-adaa-8e74993612d5 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:37.167198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 140/285] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:37.193347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f3a3-be24-458d-89f0-dcfeed2b4840 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:37.292979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e64f8-bf40-4a9f-adaa-8e74993612d5 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:37.334793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804396f-da39-4b39-addc-b3dc33449533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:37.335602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804396f-da39-4b39-addc-b3dc33449533 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:37.335894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804396f-da39-4b39-addc-b3dc33449533 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:37.336180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804396f-da39-4b39-addc-b3dc33449533 None admin] PATH_INFO: `/v3/users/6ae9b676cbb648f083e30ca48740d104` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:37.336809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:37.337769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804396f-da39-4b39-addc-b3dc33449533 None admin] RBAC: Authorizing `identity:get_user(user_id=6ae9b676cbb648f083e30ca48740d104)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:37.339266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804396f-da39-4b39-addc-b3dc33449533 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:37.341783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 146/286] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:37 2022] GET /identity/v3/users/6ae9b676cbb648f083e30ca48740d104 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:37.347029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804396f-da39-4b39-addc-b3dc33449533 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:37.368656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f3a3-be24-458d-89f0-dcfeed2b4840 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:37.410188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:37.460390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804396f-da39-4b39-addc-b3dc33449533 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:37.500850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45f6aa-8dd9-4fb3-96cb-5c2c6ffd902b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:37.502088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45f6aa-8dd9-4fb3-96cb-5c2c6ffd902b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:37.502785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45f6aa-8dd9-4fb3-96cb-5c2c6ffd902b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:37.503301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45f6aa-8dd9-4fb3-96cb-5c2c6ffd902b None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:37.504771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45f6aa-8dd9-4fb3-96cb-5c2c6ffd902b None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:37.506551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45f6aa-8dd9-4fb3-96cb-5c2c6ffd902b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:37.508766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 147/287] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:37 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:37.515874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45f6aa-8dd9-4fb3-96cb-5c2c6ffd902b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:37.517714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95863341-32f4-43de-a642-240b75648f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:37.519134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95863341-32f4-43de-a642-240b75648f0f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:37.521398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95863341-32f4-43de-a642-240b75648f0f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:37.522219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95863341-32f4-43de-a642-240b75648f0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:37.545622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95863341-32f4-43de-a642-240b75648f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-95863341-32f4-43de-a642-240b75648f0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:37.562838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:37.563759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:37.564550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:37.567722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 141/288] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:37.590402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95863341-32f4-43de-a642-240b75648f0f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:37.677611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45f6aa-8dd9-4fb3-96cb-5c2c6ffd902b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:37.719400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:37.747821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955c69-e52a-41aa-a081-18e4cd2fffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:37.750694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955c69-e52a-41aa-a081-18e4cd2fffaa None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:37.751178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955c69-e52a-41aa-a081-18e4cd2fffaa None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:37.753192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955c69-e52a-41aa-a081-18e4cd2fffaa None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f/users/6ae9b676cbb648f083e30ca48740d104/roles/eb50a4dbe6434ab0847618153c619d1a` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:37.756421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955c69-e52a-41aa-a081-18e4cd2fffaa None admin] RBAC: Authorizing `identity:create_grant(project_id=488c0084475d48098c54c5f7ee0fe92f, user_id=6ae9b676cbb648f083e30ca48740d104, role_id=eb50a4dbe6434ab0847618153c619d1a)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:37.758746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955c69-e52a-41aa-a081-18e4cd2fffaa None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:37.785264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95863341-32f4-43de-a642-240b75648f0f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:37.826828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eabc06-e7d5-44f8-8982-15b5c6301b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:37.827504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eabc06-e7d5-44f8-8982-15b5c6301b0a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:37.827800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eabc06-e7d5-44f8-8982-15b5c6301b0a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:37.828082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eabc06-e7d5-44f8-8982-15b5c6301b0a None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:37.828837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eabc06-e7d5-44f8-8982-15b5c6301b0a None admin] RBAC: Adding query filter params (name=member) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:37.829148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eabc06-e7d5-44f8-8982-15b5c6301b0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:37.830578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eabc06-e7d5-44f8-8982-15b5c6301b0a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:37.842931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 142/289] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:37.843713 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 148/290] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:34:37 2022] PUT /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f/users/6ae9b676cbb648f083e30ca48740d104/roles/eb50a4dbe6434ab0847618153c619d1a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:34:37.849710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eabc06-e7d5-44f8-8982-15b5c6301b0a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:37.878275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:38.008130 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eabc06-e7d5-44f8-8982-15b5c6301b0a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:38.067014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9beab-5a78-42d7-af96-bfeeac0addab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:38.067938 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9beab-5a78-42d7-af96-bfeeac0addab None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:38.068223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9beab-5a78-42d7-af96-bfeeac0addab None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:38.068506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9beab-5a78-42d7-af96-bfeeac0addab None admin] PATH_INFO: `/v3/users/2be2796155ec431496bdf07cdf8c4d34` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:38.070014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9beab-5a78-42d7-af96-bfeeac0addab None admin] RBAC: Authorizing `identity:get_user(user_id=2be2796155ec431496bdf07cdf8c4d34)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:38.071584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9beab-5a78-42d7-af96-bfeeac0addab None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:38.073379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 143/291] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:37 2022] GET /identity/v3/users/2be2796155ec431496bdf07cdf8c4d34 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:38.079646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955c69-e52a-41aa-a081-18e4cd2fffaa None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:38.208002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955c69-e52a-41aa-a081-18e4cd2fffaa None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:38.243700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68c952-967f-452a-a292-fb8e229c0dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:38.244362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68c952-967f-452a-a292-fb8e229c0dc9 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:38.244646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68c952-967f-452a-a292-fb8e229c0dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:38.244924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68c952-967f-452a-a292-fb8e229c0dc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:38.307307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:38.332026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68c952-967f-452a-a292-fb8e229c0dc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:38.333488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68c952-967f-452a-a292-fb8e229c0dc9 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:38.463554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 149/292] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:38.474997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9beab-5a78-42d7-af96-bfeeac0addab None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:38.580833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:38.601934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9beab-5a78-42d7-af96-bfeeac0addab None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:38.645791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5fc52-479f-4cf2-96de-43b4cd7e8574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:38.646686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5fc52-479f-4cf2-96de-43b4cd7e8574 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:38.646968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5fc52-479f-4cf2-96de-43b4cd7e8574 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:38.647244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5fc52-479f-4cf2-96de-43b4cd7e8574 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:38.654660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5fc52-479f-4cf2-96de-43b4cd7e8574 None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:38.656130 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5fc52-479f-4cf2-96de-43b4cd7e8574 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:38.657889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 144/293] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:38 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:38.663801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68c952-967f-452a-a292-fb8e229c0dc9 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:38.762325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:38.792041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68c952-967f-452a-a292-fb8e229c0dc9 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:38.842001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af7a90-34d3-41dd-a856-8bb99924029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:38.842676 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7a90-34d3-41dd-a856-8bb99924029e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:38.842962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7a90-34d3-41dd-a856-8bb99924029e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:38.843240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af7a90-34d3-41dd-a856-8bb99924029e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:38.844149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af7a90-34d3-41dd-a856-8bb99924029e None admin] RBAC: Adding query filter params (role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=488c0084475d48098c54c5f7ee0fe92f, user.id=2be2796155ec431496bdf07cdf8c4d34) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:38.844465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af7a90-34d3-41dd-a856-8bb99924029e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:38.845931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af7a90-34d3-41dd-a856-8bb99924029e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:38.852641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 150/294] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:38 2022] GET /identity/v3/role_assignments?user.id=2be2796155ec431496bdf07cdf8c4d34&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:39.230077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d45f72-03e0-4b27-a910-0609804c72b7 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.230782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d45f72-03e0-4b27-a910-0609804c72b7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.230782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d45f72-03e0-4b27-a910-0609804c72b7 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.232070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 145/295] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:39.237581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15cb87-c4f0-4caa-a52b-e6c7cfc8c9f8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.237905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15cb87-c4f0-4caa-a52b-e6c7cfc8c9f8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.238192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15cb87-c4f0-4caa-a52b-e6c7cfc8c9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.254659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a763b-c528-4255-9f4f-bb413a78dc1e None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.254969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a763b-c528-4255-9f4f-bb413a78dc1e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.255214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a763b-c528-4255-9f4f-bb413a78dc1e None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.257092 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 146/296] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:39.260960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ba15cb87-c4f0-4caa-a52b-e6c7cfc8c9f8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:39.262196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ba15cb87-c4f0-4caa-a52b-e6c7cfc8c9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:39.262391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b441a8-9d1b-4f87-b5bb-c881d45a6ed8 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.262668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b441a8-9d1b-4f87-b5bb-c881d45a6ed8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.263007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b441a8-9d1b-4f87-b5bb-c881d45a6ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.285433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-89b441a8-9d1b-4f87-b5bb-c881d45a6ed8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:39.287343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-89b441a8-9d1b-4f87-b5bb-c881d45a6ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:39.358232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 151/297] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:39.376803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 147/298] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:39.381654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15cb87-c4f0-4caa-a52b-e6c7cfc8c9f8 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:39.401476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b441a8-9d1b-4f87-b5bb-c881d45a6ed8 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:39.480411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:39.501083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:39.512210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15cb87-c4f0-4caa-a52b-e6c7cfc8c9f8 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:39.523835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b441a8-9d1b-4f87-b5bb-c881d45a6ed8 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:39.549335 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80405cb4-3fca-4dd3-817d-da52e318734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:39.550165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80405cb4-3fca-4dd3-817d-da52e318734c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.550472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80405cb4-3fca-4dd3-817d-da52e318734c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.550718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80405cb4-3fca-4dd3-817d-da52e318734c None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.561619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80405cb4-3fca-4dd3-817d-da52e318734c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:39.561885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fb64c-23f8-4ef0-9755-c7640b7ed0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:39.562685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fb64c-23f8-4ef0-9755-c7640b7ed0f7 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.562967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fb64c-23f8-4ef0-9755-c7640b7ed0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.563238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fb64c-23f8-4ef0-9755-c7640b7ed0f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-80405cb4-3fca-4dd3-817d-da52e318734c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:39.569447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:39.570367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:39.571165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 152/299] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:39.572402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f56e9-831d-4027-9d6d-e953739e20e1 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.572402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f56e9-831d-4027-9d6d-e953739e20e1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.572402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f56e9-831d-4027-9d6d-e953739e20e1 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.573766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 153/300] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:39.577168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f56e9-831d-4027-9d6d-e953739e20e1 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:39.580368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fb64c-23f8-4ef0-9755-c7640b7ed0f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-de0fb64c-23f8-4ef0-9755-c7640b7ed0f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:39.588412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:39.589481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:39.590326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 148/301] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:39.591214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201515e7-6c01-47f3-958a-7c0ca81438f5 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.591214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201515e7-6c01-47f3-958a-7c0ca81438f5 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.591399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201515e7-6c01-47f3-958a-7c0ca81438f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.608697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-201515e7-6c01-47f3-958a-7c0ca81438f5 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:39.609789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-201515e7-6c01-47f3-958a-7c0ca81438f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:39.678729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 149/302] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:39.679387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201515e7-6c01-47f3-958a-7c0ca81438f5 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:39.708745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f56e9-831d-4027-9d6d-e953739e20e1 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:39.742624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc93365-a637-4f86-b6ea-72ad10c3a223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:39.743293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc93365-a637-4f86-b6ea-72ad10c3a223 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.743573 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc93365-a637-4f86-b6ea-72ad10c3a223 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.743852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc93365-a637-4f86-b6ea-72ad10c3a223 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.744595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc93365-a637-4f86-b6ea-72ad10c3a223 None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:39.744905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc93365-a637-4f86-b6ea-72ad10c3a223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:39.746425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc93365-a637-4f86-b6ea-72ad10c3a223 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:39.747841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:39.751739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:39.759899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 154/303] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:39.759899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc93365-a637-4f86-b6ea-72ad10c3a223 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:39.791076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201515e7-6c01-47f3-958a-7c0ca81438f5 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:39.828861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062c655-8182-4cd4-b8dc-6b8244cbda7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:39.829543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062c655-8182-4cd4-b8dc-6b8244cbda7b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.829854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062c655-8182-4cd4-b8dc-6b8244cbda7b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.830223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062c655-8182-4cd4-b8dc-6b8244cbda7b None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.830982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062c655-8182-4cd4-b8dc-6b8244cbda7b None admin] RBAC: Adding query filter params (name=reader) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:39.831290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062c655-8182-4cd4-b8dc-6b8244cbda7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:39.832778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062c655-8182-4cd4-b8dc-6b8244cbda7b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:39.842361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 150/304] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:39.843167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062c655-8182-4cd4-b8dc-6b8244cbda7b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:39.870607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc93365-a637-4f86-b6ea-72ad10c3a223 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:39.907556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a8b95-2958-4297-9717-dc1d89d8a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:39.908358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a8b95-2958-4297-9717-dc1d89d8a68a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.908642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a8b95-2958-4297-9717-dc1d89d8a68a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.908918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a8b95-2958-4297-9717-dc1d89d8a68a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.910126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a8b95-2958-4297-9717-dc1d89d8a68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:39.911586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a8b95-2958-4297-9717-dc1d89d8a68a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:39.913209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 155/305] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:39.913985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a8b95-2958-4297-9717-dc1d89d8a68a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:39.959946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062c655-8182-4cd4-b8dc-6b8244cbda7b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:39.978250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:39.990530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:39.996352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59618375-8fda-4256-bc41-2dabfa372d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:39.997726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59618375-8fda-4256-bc41-2dabfa372d48 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:39.998044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59618375-8fda-4256-bc41-2dabfa372d48 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:39.998330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59618375-8fda-4256-bc41-2dabfa372d48 None admin] PATH_INFO: `/v3/users/c6fbd651e2574e5dab86a987b2460310` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:39.999567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59618375-8fda-4256-bc41-2dabfa372d48 None admin] RBAC: Authorizing `identity:get_user(user_id=c6fbd651e2574e5dab86a987b2460310)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:40.001016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59618375-8fda-4256-bc41-2dabfa372d48 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:40.007309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 151/306] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:39 2022] GET /identity/v3/users/c6fbd651e2574e5dab86a987b2460310 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:40.007309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59618375-8fda-4256-bc41-2dabfa372d48 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:40.020628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a8b95-2958-4297-9717-dc1d89d8a68a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:40.062764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6247de-98b6-4469-8424-d9fc42daabac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:40.063738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6247de-98b6-4469-8424-d9fc42daabac None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:40.064089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6247de-98b6-4469-8424-d9fc42daabac None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:40.064435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6247de-98b6-4469-8424-d9fc42daabac None admin] PATH_INFO: `/v3/users/1f64d25d84754c288568039e297726eb` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:40.163687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6247de-98b6-4469-8424-d9fc42daabac None admin] RBAC: Authorizing `identity:get_user(user_id=1f64d25d84754c288568039e297726eb)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:40.165265 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6247de-98b6-4469-8424-d9fc42daabac None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:40.167422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 156/307] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:39 2022] GET /identity/v3/users/1f64d25d84754c288568039e297726eb => generated 293 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:40.168172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6247de-98b6-4469-8424-d9fc42daabac None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:40.190628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59618375-8fda-4256-bc41-2dabfa372d48 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:40.229516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fc526-6c9a-4da1-afc9-5ab8ee68ed69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:40.230323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fc526-6c9a-4da1-afc9-5ab8ee68ed69 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:40.230852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fc526-6c9a-4da1-afc9-5ab8ee68ed69 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:40.230852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587fc526-6c9a-4da1-afc9-5ab8ee68ed69 None admin] PATH_INFO: `/v3/groups` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:40.231548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fc526-6c9a-4da1-afc9-5ab8ee68ed69 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:40.233692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587fc526-6c9a-4da1-afc9-5ab8ee68ed69 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:40.273054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6247de-98b6-4469-8424-d9fc42daabac None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:40.296623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:40.307690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cc085-7e72-44fa-8325-47765014cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:40.308567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cc085-7e72-44fa-8325-47765014cc19 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:40.308881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cc085-7e72-44fa-8325-47765014cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:40.309123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cc085-7e72-44fa-8325-47765014cc19 None admin] PATH_INFO: `/v3/system/users/c6fbd651e2574e5dab86a987b2460310/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:40.310687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cc085-7e72-44fa-8325-47765014cc19 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c6fbd651e2574e5dab86a987b2460310, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:40.312065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cc085-7e72-44fa-8325-47765014cc19 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:40.316576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 152/308] 10.222.0.47 () {60 vars in 1134 bytes} [Thu Jul 14 15:34:40 2022] POST /identity/v3/groups => generated 229 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:34:40.326990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fc526-6c9a-4da1-afc9-5ab8ee68ed69 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:40.393590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:40.417925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 157/309] 10.222.0.47 () {58 vars in 1334 bytes} [Thu Jul 14 15:34:40 2022] PUT /identity/v3/system/users/c6fbd651e2574e5dab86a987b2460310/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:40.419869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce8221-b688-4bbb-866f-305ac7c4b6d9 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:40.420161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce8221-b688-4bbb-866f-305ac7c4b6d9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:40.420443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce8221-b688-4bbb-866f-305ac7c4b6d9 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:40.422180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 158/310] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:40.428331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2a267-7874-4a62-bd8e-31718f727cea None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:40.431443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2a267-7874-4a62-bd8e-31718f727cea None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:40.431443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2a267-7874-4a62-bd8e-31718f727cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:40.448820 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-94e2a267-7874-4a62-bd8e-31718f727cea None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:40.450891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-94e2a267-7874-4a62-bd8e-31718f727cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:40.456530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587fc526-6c9a-4da1-afc9-5ab8ee68ed69 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:40.504618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56a697-876d-4acf-9a69-6fb4e9b506cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:40.505672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a697-876d-4acf-9a69-6fb4e9b506cf None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:40.506179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a697-876d-4acf-9a69-6fb4e9b506cf None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:40.506643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a697-876d-4acf-9a69-6fb4e9b506cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:40.508578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56a697-876d-4acf-9a69-6fb4e9b506cf None admin] RBAC: Adding query filter params (role.id=eb50a4dbe6434ab0847618153c619d1a, user.id=1f64d25d84754c288568039e297726eb, scope.system=all) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:40.509117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56a697-876d-4acf-9a69-6fb4e9b506cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:40.511443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56a697-876d-4acf-9a69-6fb4e9b506cf None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:40.519878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 153/311] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jul 14 15:34:40 2022] GET /identity/v3/role_assignments?user.id=1f64d25d84754c288568039e297726eb&scope.system=all&role.id=eb50a4dbe6434ab0847618153c619d1a => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:40.553928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 159/312] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:40.573447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56a697-876d-4acf-9a69-6fb4e9b506cf None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:40.700149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56a697-876d-4acf-9a69-6fb4e9b506cf None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:40.724968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:40.737266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356922ad-7726-43a3-b22c-78b9d15af262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:40.738110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356922ad-7726-43a3-b22c-78b9d15af262 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:40.740521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356922ad-7726-43a3-b22c-78b9d15af262 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:40.740521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356922ad-7726-43a3-b22c-78b9d15af262 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:40.750777 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356922ad-7726-43a3-b22c-78b9d15af262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-356922ad-7726-43a3-b22c-78b9d15af262 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:40.760358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:40.761249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:40.762074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:40.764126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 154/313] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:40.770322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2a267-7874-4a62-bd8e-31718f727cea None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:40.880669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2a267-7874-4a62-bd8e-31718f727cea None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:40.899018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:40.917227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb089d00-c9bb-479c-b0c2-fd1cd5d6ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:40.917933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb089d00-c9bb-479c-b0c2-fd1cd5d6ee2a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:40.918215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb089d00-c9bb-479c-b0c2-fd1cd5d6ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:40.918488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb089d00-c9bb-479c-b0c2-fd1cd5d6ee2a None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:40.919275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb089d00-c9bb-479c-b0c2-fd1cd5d6ee2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:40.919579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb089d00-c9bb-479c-b0c2-fd1cd5d6ee2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:40.920988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb089d00-c9bb-479c-b0c2-fd1cd5d6ee2a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:40.929864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 160/314] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:40.935908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356922ad-7726-43a3-b22c-78b9d15af262 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:41.054821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356922ad-7726-43a3-b22c-78b9d15af262 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:41.092194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76795920-8ca3-4372-a2fc-1ae20a91d8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:41.093456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76795920-8ca3-4372-a2fc-1ae20a91d8dc None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:41.093978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76795920-8ca3-4372-a2fc-1ae20a91d8dc None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:41.094453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76795920-8ca3-4372-a2fc-1ae20a91d8dc None admin] PATH_INFO: `/v3/users/6ae9b676cbb648f083e30ca48740d104` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:41.095862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76795920-8ca3-4372-a2fc-1ae20a91d8dc None admin] RBAC: Authorizing `identity:get_user(user_id=6ae9b676cbb648f083e30ca48740d104)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:41.097505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76795920-8ca3-4372-a2fc-1ae20a91d8dc None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:41.099379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 155/315] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:40 2022] GET /identity/v3/users/6ae9b676cbb648f083e30ca48740d104 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:41.105132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb089d00-c9bb-479c-b0c2-fd1cd5d6ee2a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:41.214983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb089d00-c9bb-479c-b0c2-fd1cd5d6ee2a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:41.249266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faa469-dc86-4ef9-bd5e-51eebcc6207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:41.249951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa469-dc86-4ef9-bd5e-51eebcc6207d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:41.250234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa469-dc86-4ef9-bd5e-51eebcc6207d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:41.250513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa469-dc86-4ef9-bd5e-51eebcc6207d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:41.297924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:41.319646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faa469-dc86-4ef9-bd5e-51eebcc6207d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:41.321095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faa469-dc86-4ef9-bd5e-51eebcc6207d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:41.419070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 161/316] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:41.424981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76795920-8ca3-4372-a2fc-1ae20a91d8dc None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:41.477963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:41.533720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76795920-8ca3-4372-a2fc-1ae20a91d8dc None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:41.571570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3fa07-0197-4088-b141-959563e9f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:41.572355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3fa07-0197-4088-b141-959563e9f36b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:41.572635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3fa07-0197-4088-b141-959563e9f36b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:41.572903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3fa07-0197-4088-b141-959563e9f36b None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:41.574126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3fa07-0197-4088-b141-959563e9f36b None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:41.576232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3fa07-0197-4088-b141-959563e9f36b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:41.578256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 156/317] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:41 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:41.584417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faa469-dc86-4ef9-bd5e-51eebcc6207d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:41.665555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:41.687959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faa469-dc86-4ef9-bd5e-51eebcc6207d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:41.724889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f0e32-5c6f-4d72-9874-5b813cee8396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:41.725949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f0e32-5c6f-4d72-9874-5b813cee8396 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:41.726497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f0e32-5c6f-4d72-9874-5b813cee8396 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:41.726969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f0e32-5c6f-4d72-9874-5b813cee8396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:41.728015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f0e32-5c6f-4d72-9874-5b813cee8396 None admin] RBAC: Adding query filter params (role.id=eb50a4dbe6434ab0847618153c619d1a, scope.project.id=488c0084475d48098c54c5f7ee0fe92f, user.id=6ae9b676cbb648f083e30ca48740d104) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:41.728544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f0e32-5c6f-4d72-9874-5b813cee8396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:41.730198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f0e32-5c6f-4d72-9874-5b813cee8396 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:41.741943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 162/318] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:34:41 2022] GET /identity/v3/role_assignments?user.id=6ae9b676cbb648f083e30ca48740d104&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=eb50a4dbe6434ab0847618153c619d1a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:42.510124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6c00b-2958-4057-9f03-882b7c6adde6 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.510633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6c00b-2958-4057-9f03-882b7c6adde6 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.510714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6c00b-2958-4057-9f03-882b7c6adde6 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.512272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 157/319] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:42.517535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e54da3-bc5d-44c0-9d6a-99b6e59e20a2 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.517976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e54da3-bc5d-44c0-9d6a-99b6e59e20a2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.518151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e54da3-bc5d-44c0-9d6a-99b6e59e20a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.536315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e8e54da3-bc5d-44c0-9d6a-99b6e59e20a2 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:42.537516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e8e54da3-bc5d-44c0-9d6a-99b6e59e20a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:42.567349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6d9c9-27f0-46a3-a16e-b13473b3b83d None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.567833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6d9c9-27f0-46a3-a16e-b13473b3b83d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.567938 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6d9c9-27f0-46a3-a16e-b13473b3b83d None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.569639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 158/320] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:34:42.574558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a57e9-80b1-403d-8921-907f222e931b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.574852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a57e9-80b1-403d-8921-907f222e931b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.575133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a57e9-80b1-403d-8921-907f222e931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.602821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ea6a57e9-80b1-403d-8921-907f222e931b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:42.604087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ea6a57e9-80b1-403d-8921-907f222e931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:42.622382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 163/321] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:42.639792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e54da3-bc5d-44c0-9d6a-99b6e59e20a2 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:42.685856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 159/322] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:42.685856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8d9c5-da2d-425d-b270-543f10df02d3 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.685856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8d9c5-da2d-425d-b270-543f10df02d3 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.685856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8d9c5-da2d-425d-b270-543f10df02d3 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.690151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 160/323] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:42.692489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f93fbb-2309-474e-bb8d-01ba33f562c8 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.692814 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f93fbb-2309-474e-bb8d-01ba33f562c8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.693211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f93fbb-2309-474e-bb8d-01ba33f562c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.710932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-14f93fbb-2309-474e-bb8d-01ba33f562c8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:42.712126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-14f93fbb-2309-474e-bb8d-01ba33f562c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:42.722234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:42.753281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e54da3-bc5d-44c0-9d6a-99b6e59e20a2 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:42.762351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:42.790447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 161/324] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:42.791552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f93fbb-2309-474e-bb8d-01ba33f562c8 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:42.797765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67be0-94cb-413b-88e4-216c4514fcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:42.798612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67be0-94cb-413b-88e4-216c4514fcc6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.798894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67be0-94cb-413b-88e4-216c4514fcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.799170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67be0-94cb-413b-88e4-216c4514fcc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.800435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b67be0-94cb-413b-88e4-216c4514fcc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:42.801932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b67be0-94cb-413b-88e4-216c4514fcc6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:42.804065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 164/325] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:34:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:42.809219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67be0-94cb-413b-88e4-216c4514fcc6 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:42.898571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f93fbb-2309-474e-bb8d-01ba33f562c8 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:42.911945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67be0-94cb-413b-88e4-216c4514fcc6 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:42.932363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da51cfb-0a55-47fd-9be4-80a845b1521f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:42.933156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da51cfb-0a55-47fd-9be4-80a845b1521f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.933442 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da51cfb-0a55-47fd-9be4-80a845b1521f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.933718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da51cfb-0a55-47fd-9be4-80a845b1521f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.944159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da51cfb-0a55-47fd-9be4-80a845b1521f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:42.945289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6080d85-4f50-46f2-9e4f-3c5970fe19c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:42.945980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6080d85-4f50-46f2-9e4f-3c5970fe19c8 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:42.946260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6080d85-4f50-46f2-9e4f-3c5970fe19c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:42.946532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6080d85-4f50-46f2-9e4f-3c5970fe19c8 None admin] PATH_INFO: `/v3/groups` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:42.947213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6080d85-4f50-46f2-9e4f-3c5970fe19c8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:42.948647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6080d85-4f50-46f2-9e4f-3c5970fe19c8 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-9da51cfb-0a55-47fd-9be4-80a845b1521f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:42.951878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:42.952737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:42.953540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 162/326] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:42.954954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da51cfb-0a55-47fd-9be4-80a845b1521f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:42.994574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:43.043032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:43.048320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 165/327] 10.222.0.47 () {60 vars in 1134 bytes} [Thu Jul 14 15:34:42 2022] POST /identity/v3/groups => generated 226 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:34:43.049179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6080d85-4f50-46f2-9e4f-3c5970fe19c8 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:43.056907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da51cfb-0a55-47fd-9be4-80a845b1521f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:43.097930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b0bde-5b55-49d8-883d-8030c5ca245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:43.098752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b0bde-5b55-49d8-883d-8030c5ca245f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:43.099031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b0bde-5b55-49d8-883d-8030c5ca245f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:43.099307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b0bde-5b55-49d8-883d-8030c5ca245f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:43.112051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b0bde-5b55-49d8-883d-8030c5ca245f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-9e1b0bde-5b55-49d8-883d-8030c5ca245f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:43.120011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:43.121049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:43.122165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 163/328] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:43.127744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b0bde-5b55-49d8-883d-8030c5ca245f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:43.173598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6080d85-4f50-46f2-9e4f-3c5970fe19c8 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:43.215602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2910c7-3a1c-421e-85aa-9bb654c0b6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:43.216824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2910c7-3a1c-421e-85aa-9bb654c0b6b9 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:43.217109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2910c7-3a1c-421e-85aa-9bb654c0b6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:43.217386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2910c7-3a1c-421e-85aa-9bb654c0b6b9 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:43.218177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2910c7-3a1c-421e-85aa-9bb654c0b6b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:43.218491 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2910c7-3a1c-421e-85aa-9bb654c0b6b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:43.219873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2910c7-3a1c-421e-85aa-9bb654c0b6b9 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:43.229381 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 166/329] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:43.235316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2910c7-3a1c-421e-85aa-9bb654c0b6b9 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:43.238564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:43.244206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b0bde-5b55-49d8-883d-8030c5ca245f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:43.283361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b53ded-19d8-4eca-bbd5-40bf84859934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:43.284340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b53ded-19d8-4eca-bbd5-40bf84859934 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:43.284636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b53ded-19d8-4eca-bbd5-40bf84859934 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:43.284921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b53ded-19d8-4eca-bbd5-40bf84859934 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:43.285718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b53ded-19d8-4eca-bbd5-40bf84859934 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:43.286083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b53ded-19d8-4eca-bbd5-40bf84859934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:43.287499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b53ded-19d8-4eca-bbd5-40bf84859934 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:43.298768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 164/330] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:43.303879 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b53ded-19d8-4eca-bbd5-40bf84859934 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:43.311305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:43.358051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2910c7-3a1c-421e-85aa-9bb654c0b6b9 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:43.407580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216fdab-652c-42fc-bbe5-165441e3b810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:43.408470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216fdab-652c-42fc-bbe5-165441e3b810 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:43.409257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216fdab-652c-42fc-bbe5-165441e3b810 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:43.409538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216fdab-652c-42fc-bbe5-165441e3b810 None admin] PATH_INFO: `/v3/users/c6fbd651e2574e5dab86a987b2460310` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:43.411187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216fdab-652c-42fc-bbe5-165441e3b810 None admin] RBAC: Authorizing `identity:get_user(user_id=c6fbd651e2574e5dab86a987b2460310)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:43.412955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216fdab-652c-42fc-bbe5-165441e3b810 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:43.420154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 167/331] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:43 2022] GET /identity/v3/users/c6fbd651e2574e5dab86a987b2460310 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:43.421117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216fdab-652c-42fc-bbe5-165441e3b810 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:43.443558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b53ded-19d8-4eca-bbd5-40bf84859934 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:43.483242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005bf0d-c610-4719-a1fd-f609137f18f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:43.484260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005bf0d-c610-4719-a1fd-f609137f18f4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:43.484554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005bf0d-c610-4719-a1fd-f609137f18f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:43.484837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005bf0d-c610-4719-a1fd-f609137f18f4 None admin] PATH_INFO: `/v3/users/1f64d25d84754c288568039e297726eb` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:43.486206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005bf0d-c610-4719-a1fd-f609137f18f4 None admin] RBAC: Authorizing `identity:get_user(user_id=1f64d25d84754c288568039e297726eb)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:43.487726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005bf0d-c610-4719-a1fd-f609137f18f4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:43.489767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 165/332] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:43 2022] GET /identity/v3/users/1f64d25d84754c288568039e297726eb => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:43.495727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005bf0d-c610-4719-a1fd-f609137f18f4 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:43.526579 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:43.554092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216fdab-652c-42fc-bbe5-165441e3b810 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:43.591629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677973c-6439-440a-9717-f85d288e1b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:43.592603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677973c-6439-440a-9717-f85d288e1b25 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:43.593043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677973c-6439-440a-9717-f85d288e1b25 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:43.593588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677973c-6439-440a-9717-f85d288e1b25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:43.595074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:43.595797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677973c-6439-440a-9717-f85d288e1b25 None admin] RBAC: Adding query filter params (role.id=fda37aea043b4685b930f1d9bde2b0ec, user.id=c6fbd651e2574e5dab86a987b2460310, scope.system=all) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:43.595983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677973c-6439-440a-9717-f85d288e1b25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:43.597638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677973c-6439-440a-9717-f85d288e1b25 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:43.613259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 168/333] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jul 14 15:34:43 2022] GET /identity/v3/role_assignments?user.id=c6fbd651e2574e5dab86a987b2460310&scope.system=all&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:43.627009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005bf0d-c610-4719-a1fd-f609137f18f4 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:43.669302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84a24d-d998-415f-aeb1-6e4a01392a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:43.670640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84a24d-d998-415f-aeb1-6e4a01392a76 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:43.671177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84a24d-d998-415f-aeb1-6e4a01392a76 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:43.671664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84a24d-d998-415f-aeb1-6e4a01392a76 None admin] PATH_INFO: `/v3/system/users/1f64d25d84754c288568039e297726eb/roles/eb50a4dbe6434ab0847618153c619d1a` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:43.673612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84a24d-d998-415f-aeb1-6e4a01392a76 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f64d25d84754c288568039e297726eb, role_id=eb50a4dbe6434ab0847618153c619d1a)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:43.675333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84a24d-d998-415f-aeb1-6e4a01392a76 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:43.751121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 166/334] 10.222.0.47 () {58 vars in 1334 bytes} [Thu Jul 14 15:34:43 2022] PUT /identity/v3/system/users/1f64d25d84754c288568039e297726eb/roles/eb50a4dbe6434ab0847618153c619d1a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:34:45.394085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9e43-6ccf-4f3d-a457-d43902bf3a9f None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:45.394575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9e43-6ccf-4f3d-a457-d43902bf3a9f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:45.394575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d9e43-6ccf-4f3d-a457-d43902bf3a9f None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:45.396428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 169/335] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:45.401476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481b7f8-86b2-4728-9a2f-1196eb8c81f0 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:45.402355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481b7f8-86b2-4728-9a2f-1196eb8c81f0 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:45.402841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481b7f8-86b2-4728-9a2f-1196eb8c81f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:45.423924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0481b7f8-86b2-4728-9a2f-1196eb8c81f0 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:45.425453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0481b7f8-86b2-4728-9a2f-1196eb8c81f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:45.515084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 167/336] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:45.547029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d9e43-6ccf-4f3d-a457-d43902bf3a9f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:45.669489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d9e43-6ccf-4f3d-a457-d43902bf3a9f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:45.712675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c83e53-882a-4f81-9af5-64877746a7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:45.713875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c83e53-882a-4f81-9af5-64877746a7d1 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:45.714385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c83e53-882a-4f81-9af5-64877746a7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:45.714864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c83e53-882a-4f81-9af5-64877746a7d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:45.728302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c83e53-882a-4f81-9af5-64877746a7d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-10c83e53-882a-4f81-9af5-64877746a7d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:45.736901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:45.737764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:45.738589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:45.741241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 170/337] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:45.746772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481b7f8-86b2-4728-9a2f-1196eb8c81f0 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:45.774564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:45.865946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45689f-9256-48f1-85ea-267555f3b5eb None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:45.866600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45689f-9256-48f1-85ea-267555f3b5eb None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:45.867102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45689f-9256-48f1-85ea-267555f3b5eb None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:45.867276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481b7f8-86b2-4728-9a2f-1196eb8c81f0 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:45.869286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 171/338] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:45.875632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bfb1d-c480-431f-84cc-3d0673b36ece None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:45.876199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bfb1d-c480-431f-84cc-3d0673b36ece None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:45.876683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bfb1d-c480-431f-84cc-3d0673b36ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:45.906816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-4f1bfb1d-c480-431f-84cc-3d0673b36ece None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:45.908034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4f1bfb1d-c480-431f-84cc-3d0673b36ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:45.934186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d7f15-f788-476e-82b2-07ba00c4d2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:45.935339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d7f15-f788-476e-82b2-07ba00c4d2b3 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:45.935625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d7f15-f788-476e-82b2-07ba00c4d2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:45.935904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d7f15-f788-476e-82b2-07ba00c4d2b3 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:45.936666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d7f15-f788-476e-82b2-07ba00c4d2b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:45.936978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d7f15-f788-476e-82b2-07ba00c4d2b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:45.938485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d7f15-f788-476e-82b2-07ba00c4d2b3 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:45.953410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 168/339] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:45.962599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d7f15-f788-476e-82b2-07ba00c4d2b3 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:46.006030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:46.071498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 172/340] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:46.114651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bfb1d-c480-431f-84cc-3d0673b36ece None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:46.137989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:46.164807 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d7f15-f788-476e-82b2-07ba00c4d2b3 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:46.218907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3086ca-a28d-4786-86a8-1fdd098444d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:46.219576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3086ca-a28d-4786-86a8-1fdd098444d8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:46.219863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3086ca-a28d-4786-86a8-1fdd098444d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:46.220142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3086ca-a28d-4786-86a8-1fdd098444d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:46.321497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3086ca-a28d-4786-86a8-1fdd098444d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:46.323029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3086ca-a28d-4786-86a8-1fdd098444d8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:46.402521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bfb1d-c480-431f-84cc-3d0673b36ece None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:46.430185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:46.465647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 169/341] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:34:46.467170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89c98f-97b0-4883-a40c-8929944cb56c None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:46.467463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89c98f-97b0-4883-a40c-8929944cb56c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:46.467745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89c98f-97b0-4883-a40c-8929944cb56c None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:46.472184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 170/342] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:46.472868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89c98f-97b0-4883-a40c-8929944cb56c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:46.487767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca58285-b235-48e5-a522-9dee09033fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:46.488602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca58285-b235-48e5-a522-9dee09033fa5 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:46.488885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca58285-b235-48e5-a522-9dee09033fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:46.489173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca58285-b235-48e5-a522-9dee09033fa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:46.508538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca58285-b235-48e5-a522-9dee09033fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-9ca58285-b235-48e5-a522-9dee09033fa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:46.519394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:46.520260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:46.521045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:46.524949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 173/343] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:46.526313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b1695-d3f7-4ca4-8e5c-6eda2a3d8719 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:46.526602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b1695-d3f7-4ca4-8e5c-6eda2a3d8719 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:46.526881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b1695-d3f7-4ca4-8e5c-6eda2a3d8719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:46.563281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b89b1695-d3f7-4ca4-8e5c-6eda2a3d8719 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:46.564502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b89b1695-d3f7-4ca4-8e5c-6eda2a3d8719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:46.630234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:46.691282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 174/344] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:46.692226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b1695-d3f7-4ca4-8e5c-6eda2a3d8719 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:46.698723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89c98f-97b0-4883-a40c-8929944cb56c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:46.741313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d44470-01db-4870-b680-c5b5e60c38c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:46.742142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d44470-01db-4870-b680-c5b5e60c38c1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:46.742431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d44470-01db-4870-b680-c5b5e60c38c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:46.742722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d44470-01db-4870-b680-c5b5e60c38c1 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:46.743940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d44470-01db-4870-b680-c5b5e60c38c1 None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:46.745869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d44470-01db-4870-b680-c5b5e60c38c1 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:46.751251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 171/345] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:46 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:46.752610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4138d9-3288-4289-9696-7b76913ca79e None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:46.752993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4138d9-3288-4289-9696-7b76913ca79e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:46.753283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4138d9-3288-4289-9696-7b76913ca79e None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:46.755229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 172/346] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:46.756210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4138d9-3288-4289-9696-7b76913ca79e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:46.821588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b1695-d3f7-4ca4-8e5c-6eda2a3d8719 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:46.827140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:46.862579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c012f-38a9-4a95-96e1-b7583b17b8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:46.863430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c012f-38a9-4a95-96e1-b7583b17b8b0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:46.863721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c012f-38a9-4a95-96e1-b7583b17b8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:46.864363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c012f-38a9-4a95-96e1-b7583b17b8b0 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:46.865409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c012f-38a9-4a95-96e1-b7583b17b8b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:46.865781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c012f-38a9-4a95-96e1-b7583b17b8b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:46.867301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c012f-38a9-4a95-96e1-b7583b17b8b0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:46.878605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 175/347] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:46.879832 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c012f-38a9-4a95-96e1-b7583b17b8b0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:46.893091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4138d9-3288-4289-9696-7b76913ca79e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:46.917774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:46.939593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7343f-95fb-4bf3-86df-22401b33b8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:46.941147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7343f-95fb-4bf3-86df-22401b33b8e5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:46.941632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7343f-95fb-4bf3-86df-22401b33b8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:46.942097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7343f-95fb-4bf3-86df-22401b33b8e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:46.957112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7343f-95fb-4bf3-86df-22401b33b8e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-ebe7343f-95fb-4bf3-86df-22401b33b8e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:46.966611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:46.967509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 15:34:46.968325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:46.969387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 173/348] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:46.973839 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645e977-4226-4146-a4a0-a1438e424f1e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:46.973839 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645e977-4226-4146-a4a0-a1438e424f1e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:46.973839 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645e977-4226-4146-a4a0-a1438e424f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:46.991879 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0645e977-4226-4146-a4a0-a1438e424f1e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:46.993506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0645e977-4226-4146-a4a0-a1438e424f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:47.002629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c012f-38a9-4a95-96e1-b7583b17b8b0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:47.050864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766d755-ce98-4522-9c0a-75955ca345dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:47.051744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766d755-ce98-4522-9c0a-75955ca345dd None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.052026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766d755-ce98-4522-9c0a-75955ca345dd None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.052304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766d755-ce98-4522-9c0a-75955ca345dd None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.058951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766d755-ce98-4522-9c0a-75955ca345dd None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:47.060466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766d755-ce98-4522-9c0a-75955ca345dd None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:47.063012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 176/349] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:46 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:47.063812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766d755-ce98-4522-9c0a-75955ca345dd None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:47.080128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 174/350] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:47.081014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645e977-4226-4146-a4a0-a1438e424f1e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:47.144316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:47.180100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:47.189211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766d755-ce98-4522-9c0a-75955ca345dd None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:47.220370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645e977-4226-4146-a4a0-a1438e424f1e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:47.227303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a282152-916a-408d-89b8-3224c76cdd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:47.228235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a282152-916a-408d-89b8-3224c76cdd23 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.228725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a282152-916a-408d-89b8-3224c76cdd23 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.229233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a282152-916a-408d-89b8-3224c76cdd23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.262176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f771d98-4e55-47c4-be23-5dc38dd99fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:47.264060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f771d98-4e55-47c4-be23-5dc38dd99fde None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.264573 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f771d98-4e55-47c4-be23-5dc38dd99fde None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.265054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f771d98-4e55-47c4-be23-5dc38dd99fde None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.266074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f771d98-4e55-47c4-be23-5dc38dd99fde None admin] RBAC: Adding query filter params (name=reader) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:47.267137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f771d98-4e55-47c4-be23-5dc38dd99fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:47.269075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f771d98-4e55-47c4-be23-5dc38dd99fde None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:47.282862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 175/351] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:47.283143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f771d98-4e55-47c4-be23-5dc38dd99fde None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:47.326213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a282152-916a-408d-89b8-3224c76cdd23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:47.327650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a282152-916a-408d-89b8-3224c76cdd23 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:47.415470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f771d98-4e55-47c4-be23-5dc38dd99fde None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:47.418164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:47.428181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:47.437669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 177/352] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:47.438255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a282152-916a-408d-89b8-3224c76cdd23 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:47.456166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522ead0-8dc5-4068-ac9a-b2ca485f22cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:47.457559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522ead0-8dc5-4068-ac9a-b2ca485f22cf None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.457559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522ead0-8dc5-4068-ac9a-b2ca485f22cf None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.457559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522ead0-8dc5-4068-ac9a-b2ca485f22cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.458541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522ead0-8dc5-4068-ac9a-b2ca485f22cf None admin] RBAC: Adding query filter params (group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=c89e3b28dc2a41c1822590c6a086be14) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:47.458939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522ead0-8dc5-4068-ac9a-b2ca485f22cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:47.460981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522ead0-8dc5-4068-ac9a-b2ca485f22cf None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:47.467202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 176/353] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:47 2022] GET /identity/v3/role_assignments?group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:47.468065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522ead0-8dc5-4068-ac9a-b2ca485f22cf None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:47.590460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a282152-916a-408d-89b8-3224c76cdd23 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:47.603203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522ead0-8dc5-4068-ac9a-b2ca485f22cf None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:47.631016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6442-0486-4281-bc10-c6a2543838d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:47.631946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6442-0486-4281-bc10-c6a2543838d8 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.632330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6442-0486-4281-bc10-c6a2543838d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.632611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6442-0486-4281-bc10-c6a2543838d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.646693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a6442-0486-4281-bc10-c6a2543838d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:47.648168 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fa2b5-cae4-4c4a-821e-35a5dacbfca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:47.649074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fa2b5-cae4-4c4a-821e-35a5dacbfca0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.649356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fa2b5-cae4-4c4a-821e-35a5dacbfca0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.649630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fa2b5-cae4-4c4a-821e-35a5dacbfca0 None admin] PATH_INFO: `/v3/users/1f64d25d84754c288568039e297726eb` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.650894 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fa2b5-cae4-4c4a-821e-35a5dacbfca0 None admin] RBAC: Authorizing `identity:get_user(user_id=1f64d25d84754c288568039e297726eb)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:47.652365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fa2b5-cae4-4c4a-821e-35a5dacbfca0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:47.654409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 177/354] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:34:47 2022] GET /identity/v3/users/1f64d25d84754c288568039e297726eb => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-141a6442-0486-4281-bc10-c6a2543838d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:47.654619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:47.655476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:47.656288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:47.656907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0e85c-1057-4f26-b241-33e7ebe4b16c None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.656975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 178/355] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:47.657040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0e85c-1057-4f26-b241-33e7ebe4b16c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.657040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0e85c-1057-4f26-b241-33e7ebe4b16c None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.657168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6442-0486-4281-bc10-c6a2543838d8 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:47.657984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 178/356] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:47.666731 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0e85c-1057-4f26-b241-33e7ebe4b16c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:47.713876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:47.728489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:47.787611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6442-0486-4281-bc10-c6a2543838d8 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:47.802407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f0e85c-1057-4f26-b241-33e7ebe4b16c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:47.833288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4237788-68bf-49e2-b7a5-73da1aeff8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:47.834191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4237788-68bf-49e2-b7a5-73da1aeff8fb None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.834476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4237788-68bf-49e2-b7a5-73da1aeff8fb None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.834754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4237788-68bf-49e2-b7a5-73da1aeff8fb None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.835937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4237788-68bf-49e2-b7a5-73da1aeff8fb None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:47.837373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4237788-68bf-49e2-b7a5-73da1aeff8fb None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:47.839266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 179/357] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:47 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:47.843883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4237788-68bf-49e2-b7a5-73da1aeff8fb None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:47.858096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b178e1-b3dd-4d70-998b-55f9416adba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:47.858939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b178e1-b3dd-4d70-998b-55f9416adba5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.859593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b178e1-b3dd-4d70-998b-55f9416adba5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.859890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b178e1-b3dd-4d70-998b-55f9416adba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.860737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b178e1-b3dd-4d70-998b-55f9416adba5 None admin] RBAC: Adding query filter params (role.id=eb50a4dbe6434ab0847618153c619d1a, user.id=1f64d25d84754c288568039e297726eb, scope.system=all) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:47.861056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b178e1-b3dd-4d70-998b-55f9416adba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:47.863317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b178e1-b3dd-4d70-998b-55f9416adba5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:47.884987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 179/358] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jul 14 15:34:47 2022] GET /identity/v3/role_assignments?user.id=1f64d25d84754c288568039e297726eb&scope.system=all&role.id=eb50a4dbe6434ab0847618153c619d1a => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:47.893848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a2d4f-2ed6-4ad6-9c14-0011560e8ca7 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:47.894614 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a2d4f-2ed6-4ad6-9c14-0011560e8ca7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:47.895107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a2d4f-2ed6-4ad6-9c14-0011560e8ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:47.916838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-682a2d4f-2ed6-4ad6-9c14-0011560e8ca7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:47.918174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-682a2d4f-2ed6-4ad6-9c14-0011560e8ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:47.969570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4237788-68bf-49e2-b7a5-73da1aeff8fb None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:47.998581 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 180/359] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:47.999804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:48.000136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4db65-19f5-4dbc-bd5b-ed57b1726c85 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.000768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4db65-19f5-4dbc-bd5b-ed57b1726c85 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.000768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4db65-19f5-4dbc-bd5b-ed57b1726c85 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.003177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 181/360] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:48.003779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4db65-19f5-4dbc-bd5b-ed57b1726c85 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:48.014022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb12a28-d9d2-473f-bf9e-560085fd1416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:48.014984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb12a28-d9d2-473f-bf9e-560085fd1416 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.015459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb12a28-d9d2-473f-bf9e-560085fd1416 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.015875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb12a28-d9d2-473f-bf9e-560085fd1416 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.016876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb12a28-d9d2-473f-bf9e-560085fd1416 None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:48.017340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb12a28-d9d2-473f-bf9e-560085fd1416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:48.018888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb12a28-d9d2-473f-bf9e-560085fd1416 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:48.030008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 180/361] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:48.031696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0080d9-379c-48af-86c2-a3bd4692ecd4 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.032155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0080d9-379c-48af-86c2-a3bd4692ecd4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.032629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0080d9-379c-48af-86c2-a3bd4692ecd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.041365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:48.062691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cd0080d9-379c-48af-86c2-a3bd4692ecd4 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:48.064297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cd0080d9-379c-48af-86c2-a3bd4692ecd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:48.137748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4db65-19f5-4dbc-bd5b-ed57b1726c85 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:48.148632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 181/362] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:48.149491 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0080d9-379c-48af-86c2-a3bd4692ecd4 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:48.185266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6bb9b-8a11-48da-9516-6e6f07efc4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:48.186804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6bb9b-8a11-48da-9516-6e6f07efc4c7 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.187096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6bb9b-8a11-48da-9516-6e6f07efc4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.187376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6bb9b-8a11-48da-9516-6e6f07efc4c7 None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.188604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6bb9b-8a11-48da-9516-6e6f07efc4c7 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:48.190126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6bb9b-8a11-48da-9516-6e6f07efc4c7 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:48.192178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 182/363] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:48 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:48.192802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6bb9b-8a11-48da-9516-6e6f07efc4c7 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:48.303893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0080d9-379c-48af-86c2-a3bd4692ecd4 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:48.315048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:48.322630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6bb9b-8a11-48da-9516-6e6f07efc4c7 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:48.324104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:48.360851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01db29-477a-4a4d-821a-cd2641646df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:48.361985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01db29-477a-4a4d-821a-cd2641646df0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.361985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01db29-477a-4a4d-821a-cd2641646df0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.362173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01db29-477a-4a4d-821a-cd2641646df0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.370203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c100ba-7896-4529-aeda-8992b617066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:48.371283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c100ba-7896-4529-aeda-8992b617066f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.371748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c100ba-7896-4529-aeda-8992b617066f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.372693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c100ba-7896-4529-aeda-8992b617066f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.384876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c100ba-7896-4529-aeda-8992b617066f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-c1c100ba-7896-4529-aeda-8992b617066f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:48.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:48.394860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:48.395869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 183/364] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:48.396696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c100ba-7896-4529-aeda-8992b617066f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:48.455161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01db29-477a-4a4d-821a-cd2641646df0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:48.457488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01db29-477a-4a4d-821a-cd2641646df0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:48.546247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c100ba-7896-4529-aeda-8992b617066f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:48.589300 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300b032-6d12-449b-8483-3659af87bfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:48.589935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300b032-6d12-449b-8483-3659af87bfbe None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.590469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300b032-6d12-449b-8483-3659af87bfbe None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.590961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300b032-6d12-449b-8483-3659af87bfbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.593650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 182/365] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:48.594718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01db29-477a-4a4d-821a-cd2641646df0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:48.606169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300b032-6d12-449b-8483-3659af87bfbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:48.612251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-1300b032-6d12-449b-8483-3659af87bfbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:48.612707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:48.613631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:48.614509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 184/366] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:48.615178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300b032-6d12-449b-8483-3659af87bfbe None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:48.636842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:48.736922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01db29-477a-4a4d-821a-cd2641646df0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:48.748340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300b032-6d12-449b-8483-3659af87bfbe None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:48.783094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740e742-57d6-496e-8c37-e03fa324e7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:48.783860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740e742-57d6-496e-8c37-e03fa324e7f3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.784263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740e742-57d6-496e-8c37-e03fa324e7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.784546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740e742-57d6-496e-8c37-e03fa324e7f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.785386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740e742-57d6-496e-8c37-e03fa324e7f3 None admin] RBAC: Adding query filter params (group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=c89e3b28dc2a41c1822590c6a086be14) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:48.785697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740e742-57d6-496e-8c37-e03fa324e7f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:48.787398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740e742-57d6-496e-8c37-e03fa324e7f3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:48.793356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 183/367] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:48 2022] GET /identity/v3/role_assignments?group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:48.794859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740e742-57d6-496e-8c37-e03fa324e7f3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:48.804067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360c814-f003-4839-9070-59493d66a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:48.804787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360c814-f003-4839-9070-59493d66a322 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.805737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360c814-f003-4839-9070-59493d66a322 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.806063 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360c814-f003-4839-9070-59493d66a322 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.806854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360c814-f003-4839-9070-59493d66a322 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:48.807171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360c814-f003-4839-9070-59493d66a322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:48.809505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360c814-f003-4839-9070-59493d66a322 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:48.823039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 185/368] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:48.823931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360c814-f003-4839-9070-59493d66a322 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:48.910549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:48.931107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:48.938351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740e742-57d6-496e-8c37-e03fa324e7f3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:48.959202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360c814-f003-4839-9070-59493d66a322 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:48.989308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503518f-aef2-4298-9979-f6cbe8b0c2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:48.991005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503518f-aef2-4298-9979-f6cbe8b0c2ec None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:48.991745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503518f-aef2-4298-9979-f6cbe8b0c2ec None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:48.992235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503518f-aef2-4298-9979-f6cbe8b0c2ec None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:48.994236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503518f-aef2-4298-9979-f6cbe8b0c2ec None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:48.996395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503518f-aef2-4298-9979-f6cbe8b0c2ec None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:48.999917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 184/369] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:48 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:49.001090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503518f-aef2-4298-9979-f6cbe8b0c2ec None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:49.010785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189d3d2-8e41-4896-9e7d-840582440e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:49.010785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189d3d2-8e41-4896-9e7d-840582440e0a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.010785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189d3d2-8e41-4896-9e7d-840582440e0a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.010785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189d3d2-8e41-4896-9e7d-840582440e0a None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:49.011173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189d3d2-8e41-4896-9e7d-840582440e0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:49.011173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189d3d2-8e41-4896-9e7d-840582440e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:49.011173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189d3d2-8e41-4896-9e7d-840582440e0a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:49.020127 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 186/370] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:49.020808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189d3d2-8e41-4896-9e7d-840582440e0a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:49.131911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503518f-aef2-4298-9979-f6cbe8b0c2ec None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:49.146078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189d3d2-8e41-4896-9e7d-840582440e0a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:49.175261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bbe5f-98dc-45c3-8d40-5eefd564c222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:49.175924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bbe5f-98dc-45c3-8d40-5eefd564c222 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.176206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bbe5f-98dc-45c3-8d40-5eefd564c222 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.176478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bbe5f-98dc-45c3-8d40-5eefd564c222 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:49.188104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d7cfe-02c7-4334-b7d8-0396cf63a597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:49.188977 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d7cfe-02c7-4334-b7d8-0396cf63a597 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.189266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d7cfe-02c7-4334-b7d8-0396cf63a597 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.189544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d7cfe-02c7-4334-b7d8-0396cf63a597 None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:49.191155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d7cfe-02c7-4334-b7d8-0396cf63a597 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:49.193681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d7cfe-02c7-4334-b7d8-0396cf63a597 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:49.200337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 187/371] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:49 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:49.201216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d7cfe-02c7-4334-b7d8-0396cf63a597 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:49.205013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:49.223055 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:49.263374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bbe5f-98dc-45c3-8d40-5eefd564c222 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:49.265183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bbe5f-98dc-45c3-8d40-5eefd564c222 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:49.328913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d7cfe-02c7-4334-b7d8-0396cf63a597 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:49.371888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cf3a8-414b-4309-8469-441fa158942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:49.372615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cf3a8-414b-4309-8469-441fa158942e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.372901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cf3a8-414b-4309-8469-441fa158942e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.373180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cf3a8-414b-4309-8469-441fa158942e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:49.399952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 185/372] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:49.400636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bbe5f-98dc-45c3-8d40-5eefd564c222 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:49.460260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:49.469506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cf3a8-414b-4309-8469-441fa158942e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:49.471778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cf3a8-414b-4309-8469-441fa158942e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:49.475356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:49.522442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bbe5f-98dc-45c3-8d40-5eefd564c222 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:49.564057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e639a3-bd70-4a6d-b653-e3655ff2fe80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:49.564726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e639a3-bd70-4a6d-b653-e3655ff2fe80 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.565010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e639a3-bd70-4a6d-b653-e3655ff2fe80 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.565290 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e639a3-bd70-4a6d-b653-e3655ff2fe80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:49.566436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e639a3-bd70-4a6d-b653-e3655ff2fe80 None admin] RBAC: Adding query filter params (group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=488c0084475d48098c54c5f7ee0fe92f) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:49.566750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e639a3-bd70-4a6d-b653-e3655ff2fe80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:49.568189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e639a3-bd70-4a6d-b653-e3655ff2fe80 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:49.574385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 186/373] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:49 2022] GET /identity/v3/role_assignments?group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:49.582504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e639a3-bd70-4a6d-b653-e3655ff2fe80 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:49.604083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 188/374] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:49.610797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cf3a8-414b-4309-8469-441fa158942e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:49.745379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cf3a8-414b-4309-8469-441fa158942e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:49.748176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:49.800650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96996652-2b7b-4e0a-84bf-1eee10c55bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:49.802251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96996652-2b7b-4e0a-84bf-1eee10c55bae None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.802546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96996652-2b7b-4e0a-84bf-1eee10c55bae None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.802828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96996652-2b7b-4e0a-84bf-1eee10c55bae None admin] PATH_INFO: `/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:49.804685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96996652-2b7b-4e0a-84bf-1eee10c55bae None admin] RBAC: Authorizing `identity:get_project(project_id=d8c317cfa3fe4e0aa6bdd74dbc68a586)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:49.806537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96996652-2b7b-4e0a-84bf-1eee10c55bae None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:49.807358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e639a3-bd70-4a6d-b653-e3655ff2fe80 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:49.808668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 189/375] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:49 2022] GET /identity/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:49.811384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e576-ac3c-45d5-9c7a-b87eacbfb2be None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.811674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e576-ac3c-45d5-9c7a-b87eacbfb2be None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.811955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2e576-ac3c-45d5-9c7a-b87eacbfb2be None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:49.813899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 190/376] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:49.819445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e576-ac3c-45d5-9c7a-b87eacbfb2be None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:49.834225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:49.855619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5db8d8-9276-4e3d-bd0b-fa28d8e61f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:49.856557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5db8d8-9276-4e3d-bd0b-fa28d8e61f6f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.856843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5db8d8-9276-4e3d-bd0b-fa28d8e61f6f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.857107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5db8d8-9276-4e3d-bd0b-fa28d8e61f6f None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:49.859216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5db8d8-9276-4e3d-bd0b-fa28d8e61f6f None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:49.860041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5db8d8-9276-4e3d-bd0b-fa28d8e61f6f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:49.861932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 187/377] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:49 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:49.863015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3c9e8-9280-4c6f-8085-e6ae4e7ea6b9 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.863493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3c9e8-9280-4c6f-8085-e6ae4e7ea6b9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.863676 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3c9e8-9280-4c6f-8085-e6ae4e7ea6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:49.884979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cfc3c9e8-9280-4c6f-8085-e6ae4e7ea6b9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:49.886229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cfc3c9e8-9280-4c6f-8085-e6ae4e7ea6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:49.957934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2e576-ac3c-45d5-9c7a-b87eacbfb2be None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:49.980337 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 188/378] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:49.981219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3c9e8-9280-4c6f-8085-e6ae4e7ea6b9 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:49.995737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a2c5a-3dff-4357-890d-5488190572af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:49.996608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a2c5a-3dff-4357-890d-5488190572af None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:49.996890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a2c5a-3dff-4357-890d-5488190572af None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:49.997707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a2c5a-3dff-4357-890d-5488190572af None admin] PATH_INFO: `/v3/groups/52b9012085a9417d94d28d7ee872c4ab` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:50.005096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a2c5a-3dff-4357-890d-5488190572af None admin] RBAC: Authorizing `identity:get_group(group_id=52b9012085a9417d94d28d7ee872c4ab)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:50.006658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a2c5a-3dff-4357-890d-5488190572af None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:50.008888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 191/379] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:49 2022] GET /identity/v3/groups/52b9012085a9417d94d28d7ee872c4ab => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:50.015212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a2c5a-3dff-4357-890d-5488190572af None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:50.090319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:50.122049 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3c9e8-9280-4c6f-8085-e6ae4e7ea6b9 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:50.134944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:50.173486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a2c5a-3dff-4357-890d-5488190572af None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:50.174665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900a5e9-7e21-4d6b-a901-9650f5e7a99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:50.175860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900a5e9-7e21-4d6b-a901-9650f5e7a99c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:50.176150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900a5e9-7e21-4d6b-a901-9650f5e7a99c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:50.176434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900a5e9-7e21-4d6b-a901-9650f5e7a99c None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:50.179642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900a5e9-7e21-4d6b-a901-9650f5e7a99c None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:50.181216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900a5e9-7e21-4d6b-a901-9650f5e7a99c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:50.183285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 189/380] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:49 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:50.184067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900a5e9-7e21-4d6b-a901-9650f5e7a99c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:50.212071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd392d-8a1c-4b8d-8032-909aa94ee936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:50.212745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd392d-8a1c-4b8d-8032-909aa94ee936 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:50.213032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd392d-8a1c-4b8d-8032-909aa94ee936 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:50.213308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd392d-8a1c-4b8d-8032-909aa94ee936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:50.214298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd392d-8a1c-4b8d-8032-909aa94ee936 None admin] RBAC: Adding query filter params (group.id=52b9012085a9417d94d28d7ee872c4ab, role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=d8c317cfa3fe4e0aa6bdd74dbc68a586) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:50.214615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd392d-8a1c-4b8d-8032-909aa94ee936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:50.216088 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd392d-8a1c-4b8d-8032-909aa94ee936 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:50.223434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 192/381] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:50 2022] GET /identity/v3/role_assignments?group.id=52b9012085a9417d94d28d7ee872c4ab&scope.project.id=d8c317cfa3fe4e0aa6bdd74dbc68a586&role.id=abfb60dccca7470295258595a66a5d17 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:50.226645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd392d-8a1c-4b8d-8032-909aa94ee936 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:50.319097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900a5e9-7e21-4d6b-a901-9650f5e7a99c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:50.361954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87d02d-6f40-49b2-a6d4-284fcfe74040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:50.363383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87d02d-6f40-49b2-a6d4-284fcfe74040 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:50.363878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87d02d-6f40-49b2-a6d4-284fcfe74040 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:50.364795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87d02d-6f40-49b2-a6d4-284fcfe74040 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:50.378293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87d02d-6f40-49b2-a6d4-284fcfe74040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-ef87d02d-6f40-49b2-a6d4-284fcfe74040 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:50.389458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:50.390572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:50.391424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:50.393960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 190/382] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:50.400921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87d02d-6f40-49b2-a6d4-284fcfe74040 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:50.437795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:50.440609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd392d-8a1c-4b8d-8032-909aa94ee936 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:50.443858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:50.485220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86c16e-75db-4557-a346-0e3856b0fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:50.485934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86c16e-75db-4557-a346-0e3856b0fbfd None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:50.486218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86c16e-75db-4557-a346-0e3856b0fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:50.486499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86c16e-75db-4557-a346-0e3856b0fbfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:50.487408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86c16e-75db-4557-a346-0e3856b0fbfd None admin] RBAC: Adding query filter params (group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=488c0084475d48098c54c5f7ee0fe92f) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:50.487725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86c16e-75db-4557-a346-0e3856b0fbfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:50.489186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86c16e-75db-4557-a346-0e3856b0fbfd None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:50.498136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 193/383] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:50 2022] GET /identity/v3/role_assignments?group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 260 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:50.525120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87d02d-6f40-49b2-a6d4-284fcfe74040 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:50.569667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef2d46-e416-4e41-b6a1-15bd4c3737ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:50.570397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef2d46-e416-4e41-b6a1-15bd4c3737ff None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:50.570678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef2d46-e416-4e41-b6a1-15bd4c3737ff None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:50.570950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef2d46-e416-4e41-b6a1-15bd4c3737ff None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:50.571786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef2d46-e416-4e41-b6a1-15bd4c3737ff None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:50.572094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef2d46-e416-4e41-b6a1-15bd4c3737ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:50.573589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef2d46-e416-4e41-b6a1-15bd4c3737ff None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:50.586155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 191/384] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:50.592104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86c16e-75db-4557-a346-0e3856b0fbfd None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:50.743991 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86c16e-75db-4557-a346-0e3856b0fbfd None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:50.786683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b58352-c8ed-4b45-bf3e-3803ca329bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:50.787924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b58352-c8ed-4b45-bf3e-3803ca329bd5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:50.788423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b58352-c8ed-4b45-bf3e-3803ca329bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:50.788899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b58352-c8ed-4b45-bf3e-3803ca329bd5 None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:50.790304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b58352-c8ed-4b45-bf3e-3803ca329bd5 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:50.791970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b58352-c8ed-4b45-bf3e-3803ca329bd5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:50.793881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 194/385] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:50 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:50.799645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef2d46-e416-4e41-b6a1-15bd4c3737ff None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:50.910294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:50.916992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef2d46-e416-4e41-b6a1-15bd4c3737ff None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:50.957291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbae17-217c-48f4-b992-8678bf88cb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:50.958861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbae17-217c-48f4-b992-8678bf88cb7d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:50.959433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbae17-217c-48f4-b992-8678bf88cb7d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:50.960090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbae17-217c-48f4-b992-8678bf88cb7d None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:50.961606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecbae17-217c-48f4-b992-8678bf88cb7d None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:50.963089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecbae17-217c-48f4-b992-8678bf88cb7d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:50.965110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 192/386] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:50 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:50.970878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b58352-c8ed-4b45-bf3e-3803ca329bd5 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:51.033108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:51.136612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b58352-c8ed-4b45-bf3e-3803ca329bd5 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:51.180557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451b3eb-eb61-4086-9734-93e707ca5dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:51.181618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451b3eb-eb61-4086-9734-93e707ca5dfa None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:51.181940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451b3eb-eb61-4086-9734-93e707ca5dfa None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:51.182224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451b3eb-eb61-4086-9734-93e707ca5dfa None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:51.184207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451b3eb-eb61-4086-9734-93e707ca5dfa None admin] RBAC: Authorizing `identity:create_grant(project_id=c89e3b28dc2a41c1822590c6a086be14, group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:51.186234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451b3eb-eb61-4086-9734-93e707ca5dfa None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:51.256259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 195/387] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 14 15:34:50 2022] PUT /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:51.337542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655564f-4053-4365-9c4a-845f26a6c773 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:51.337861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655564f-4053-4365-9c4a-845f26a6c773 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:51.338065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655564f-4053-4365-9c4a-845f26a6c773 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:51.339792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 193/388] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:51.346034 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25757463-ac6c-458d-8cc1-2a0a8c887fc6 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:51.346330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25757463-ac6c-458d-8cc1-2a0a8c887fc6 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:51.346671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25757463-ac6c-458d-8cc1-2a0a8c887fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:51.364525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-25757463-ac6c-458d-8cc1-2a0a8c887fc6 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:51.365780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-25757463-ac6c-458d-8cc1-2a0a8c887fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:51.441694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 196/389] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:51.489974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655564f-4053-4365-9c4a-845f26a6c773 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:51.607414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655564f-4053-4365-9c4a-845f26a6c773 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:51.650252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3821859-ebe3-43e5-b9e7-32a88b42ead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:51.651075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3821859-ebe3-43e5-b9e7-32a88b42ead9 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:51.651354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3821859-ebe3-43e5-b9e7-32a88b42ead9 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:51.651627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3821859-ebe3-43e5-b9e7-32a88b42ead9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:51.664404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3821859-ebe3-43e5-b9e7-32a88b42ead9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-c3821859-ebe3-43e5-b9e7-32a88b42ead9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:51.672576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:51.673509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:51.674349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:51.675449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 194/390] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:51.681178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25757463-ac6c-458d-8cc1-2a0a8c887fc6 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:51.748725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:51.828595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25757463-ac6c-458d-8cc1-2a0a8c887fc6 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:51.868544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c938504-7432-4e30-874a-00d93d60cf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:51.869140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c938504-7432-4e30-874a-00d93d60cf04 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:51.869429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c938504-7432-4e30-874a-00d93d60cf04 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:51.869707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c938504-7432-4e30-874a-00d93d60cf04 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:51.870669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c938504-7432-4e30-874a-00d93d60cf04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:51.870982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c938504-7432-4e30-874a-00d93d60cf04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:51.872964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c938504-7432-4e30-874a-00d93d60cf04 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:51.882263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 197/391] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:51.887565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3821859-ebe3-43e5-b9e7-32a88b42ead9 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:51.935570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:52.035757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3821859-ebe3-43e5-b9e7-32a88b42ead9 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:52.085332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3a320-fe53-4e1d-a522-3a05213bbef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:52.086235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3a320-fe53-4e1d-a522-3a05213bbef6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:52.086514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3a320-fe53-4e1d-a522-3a05213bbef6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:52.086788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3a320-fe53-4e1d-a522-3a05213bbef6 None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:52.087985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3a320-fe53-4e1d-a522-3a05213bbef6 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:52.089429 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3a320-fe53-4e1d-a522-3a05213bbef6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:52.091116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 195/392] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:51 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:52.097010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c938504-7432-4e30-874a-00d93d60cf04 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:52.210370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:52.229881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c938504-7432-4e30-874a-00d93d60cf04 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:52.276825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d49ce2-8f55-4c45-bb36-4a431b92bc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:52.277702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d49ce2-8f55-4c45-bb36-4a431b92bc9b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:52.278019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d49ce2-8f55-4c45-bb36-4a431b92bc9b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:52.278829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d49ce2-8f55-4c45-bb36-4a431b92bc9b None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:52.280032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d49ce2-8f55-4c45-bb36-4a431b92bc9b None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:52.281644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d49ce2-8f55-4c45-bb36-4a431b92bc9b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:52.283703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 198/393] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:52 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:52.289077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3a320-fe53-4e1d-a522-3a05213bbef6 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:52.409428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:52.420496 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3a320-fe53-4e1d-a522-3a05213bbef6 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:52.469459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b6924-ec54-4b1a-ac8b-6c9565109591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:52.470379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b6924-ec54-4b1a-ac8b-6c9565109591 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:52.470661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b6924-ec54-4b1a-ac8b-6c9565109591 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:52.470936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b6924-ec54-4b1a-ac8b-6c9565109591 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54/roles/2806b70c57be44f9b260ff5322bf02fb` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:52.478599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b6924-ec54-4b1a-ac8b-6c9565109591 None admin] RBAC: Authorizing `identity:create_grant(project_id=c89e3b28dc2a41c1822590c6a086be14, group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role_id=2806b70c57be44f9b260ff5322bf02fb)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:52.480057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b6924-ec54-4b1a-ac8b-6c9565109591 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:52.550701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 196/394] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 14 15:34:52 2022] PUT /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54/roles/2806b70c57be44f9b260ff5322bf02fb => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:52.621532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdb27e-cf7a-4494-91f3-1d88ad7c3c9d None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:52.622007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdb27e-cf7a-4494-91f3-1d88ad7c3c9d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:52.622756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdb27e-cf7a-4494-91f3-1d88ad7c3c9d None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:52.624450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 199/395] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:52.630302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d3dd9-bdb5-4d4e-a722-9b9651a1d50a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:52.630809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d3dd9-bdb5-4d4e-a722-9b9651a1d50a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:52.630908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d3dd9-bdb5-4d4e-a722-9b9651a1d50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:52.654968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7e2d3dd9-bdb5-4d4e-a722-9b9651a1d50a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:52.656226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7e2d3dd9-bdb5-4d4e-a722-9b9651a1d50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:52.748993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 197/396] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:52.770972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdb27e-cf7a-4494-91f3-1d88ad7c3c9d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:52.795962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97573d-430e-4cfc-9976-e5212bf2f1f1 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:52.796278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97573d-430e-4cfc-9976-e5212bf2f1f1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:52.796532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97573d-430e-4cfc-9976-e5212bf2f1f1 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:52.798197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 198/397] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:52.803519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265749c-2bb2-4807-9a64-c2714bc3bd47 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:52.803812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265749c-2bb2-4807-9a64-c2714bc3bd47 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:52.804090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265749c-2bb2-4807-9a64-c2714bc3bd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:52.825940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-4265749c-2bb2-4807-9a64-c2714bc3bd47 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:52.827285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4265749c-2bb2-4807-9a64-c2714bc3bd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:52.902882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 199/398] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:52.927066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265749c-2bb2-4807-9a64-c2714bc3bd47 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:52.927397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdb27e-cf7a-4494-91f3-1d88ad7c3c9d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:52.946467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:52.994447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfc163-f744-42aa-b302-5997c606f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:52.995268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfc163-f744-42aa-b302-5997c606f02b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:52.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfc163-f744-42aa-b302-5997c606f02b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:52.995801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acfc163-f744-42aa-b302-5997c606f02b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.015733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acfc163-f744-42aa-b302-5997c606f02b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-9acfc163-f744-42aa-b302-5997c606f02b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:53.023483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:53.029605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:53.030462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 200/399] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:53.031622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfc163-f744-42aa-b302-5997c606f02b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:53.062850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:53.100967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265749c-2bb2-4807-9a64-c2714bc3bd47 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:53.167642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279076a-0c91-4ad5-91df-959a6475b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:53.168444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279076a-0c91-4ad5-91df-959a6475b6bf None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.168729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279076a-0c91-4ad5-91df-959a6475b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.169027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279076a-0c91-4ad5-91df-959a6475b6bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.187112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c279076a-0c91-4ad5-91df-959a6475b6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-c279076a-0c91-4ad5-91df-959a6475b6bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:53.196123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:53.197031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:53.197856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:53.201305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 200/400] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:53.207768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279076a-0c91-4ad5-91df-959a6475b6bf None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:53.223844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acfc163-f744-42aa-b302-5997c606f02b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:53.307533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad97f1-8574-4522-b541-e4c38a22a966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:53.308246 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad97f1-8574-4522-b541-e4c38a22a966 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.308602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad97f1-8574-4522-b541-e4c38a22a966 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.308955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad97f1-8574-4522-b541-e4c38a22a966 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.309859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad97f1-8574-4522-b541-e4c38a22a966 None admin] RBAC: Adding query filter params (name=member) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:53.310232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad97f1-8574-4522-b541-e4c38a22a966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:53.311758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad97f1-8574-4522-b541-e4c38a22a966 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:53.318725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:53.322535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 201/401] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:53.324165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60932b93-ee81-495a-bb1e-4e8e139ed62d None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.324666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60932b93-ee81-495a-bb1e-4e8e139ed62d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.325707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60932b93-ee81-495a-bb1e-4e8e139ed62d None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.327352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 202/402] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:53.334825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60932b93-ee81-495a-bb1e-4e8e139ed62d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:53.348047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c279076a-0c91-4ad5-91df-959a6475b6bf None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:53.382927 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98436d7-0026-42f9-a8cf-d29230d6e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:53.383654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98436d7-0026-42f9-a8cf-d29230d6e61d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.383940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98436d7-0026-42f9-a8cf-d29230d6e61d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.384218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98436d7-0026-42f9-a8cf-d29230d6e61d None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.384980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98436d7-0026-42f9-a8cf-d29230d6e61d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:53.385287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98436d7-0026-42f9-a8cf-d29230d6e61d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:53.386691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98436d7-0026-42f9-a8cf-d29230d6e61d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:53.397005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 201/403] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:53.398415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52df55-bd66-45e0-8481-fc4ad1c395e4 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.398674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52df55-bd66-45e0-8481-fc4ad1c395e4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.398994 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52df55-bd66-45e0-8481-fc4ad1c395e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.417808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ad52df55-bd66-45e0-8481-fc4ad1c395e4 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:53.419060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ad52df55-bd66-45e0-8481-fc4ad1c395e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:53.432748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:53.467994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60932b93-ee81-495a-bb1e-4e8e139ed62d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:53.501135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 202/404] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:53.501135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52df55-bd66-45e0-8481-fc4ad1c395e4 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:53.535201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da8396-b617-4831-92f5-8078449c7a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:53.536500 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da8396-b617-4831-92f5-8078449c7a85 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.537003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da8396-b617-4831-92f5-8078449c7a85 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.537464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da8396-b617-4831-92f5-8078449c7a85 None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.538848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da8396-b617-4831-92f5-8078449c7a85 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:53.540524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da8396-b617-4831-92f5-8078449c7a85 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:53.542881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 203/405] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:53 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:53.543947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da8396-b617-4831-92f5-8078449c7a85 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:53.578923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:53.625243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52df55-bd66-45e0-8481-fc4ad1c395e4 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:53.655674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da8396-b617-4831-92f5-8078449c7a85 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:53.669343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441430e5-261e-4fdd-91ff-6d57da010d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:53.670249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441430e5-261e-4fdd-91ff-6d57da010d1b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.670536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441430e5-261e-4fdd-91ff-6d57da010d1b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.670812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441430e5-261e-4fdd-91ff-6d57da010d1b None admin] PATH_INFO: `/v3/groups/52b9012085a9417d94d28d7ee872c4ab` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.672184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441430e5-261e-4fdd-91ff-6d57da010d1b None admin] RBAC: Authorizing `identity:get_group(group_id=52b9012085a9417d94d28d7ee872c4ab)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:53.674391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441430e5-261e-4fdd-91ff-6d57da010d1b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:53.676902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 203/406] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:53 2022] GET /identity/v3/groups/52b9012085a9417d94d28d7ee872c4ab => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:53.677769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441430e5-261e-4fdd-91ff-6d57da010d1b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:53.697021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7844cf2-188c-4022-b72c-03116f173073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:53.698303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7844cf2-188c-4022-b72c-03116f173073 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.698919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7844cf2-188c-4022-b72c-03116f173073 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.699388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7844cf2-188c-4022-b72c-03116f173073 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.711890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7844cf2-188c-4022-b72c-03116f173073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-b7844cf2-188c-4022-b72c-03116f173073 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:53.721298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:53.722209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:53.723002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:53.730139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 204/407] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:53.731262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7844cf2-188c-4022-b72c-03116f173073 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:53.735113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:53.796064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441430e5-261e-4fdd-91ff-6d57da010d1b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:53.835504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fda304-65b8-4449-9de4-7ae4248d8b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:53.836732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda304-65b8-4449-9de4-7ae4248d8b1d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.837218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda304-65b8-4449-9de4-7ae4248d8b1d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.837800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda304-65b8-4449-9de4-7ae4248d8b1d None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.839300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fda304-65b8-4449-9de4-7ae4248d8b1d None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:53.841177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fda304-65b8-4449-9de4-7ae4248d8b1d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:53.843592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 204/408] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:53 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:53.844737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fda304-65b8-4449-9de4-7ae4248d8b1d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:53.848131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:53.862245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7844cf2-188c-4022-b72c-03116f173073 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:53.898574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483f279-c280-42f6-b757-f17f621a78ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:53.899553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483f279-c280-42f6-b757-f17f621a78ed None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:53.899838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483f279-c280-42f6-b757-f17f621a78ed None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:53.900115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483f279-c280-42f6-b757-f17f621a78ed None admin] PATH_INFO: `/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:53.901393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483f279-c280-42f6-b757-f17f621a78ed None admin] RBAC: Authorizing `identity:get_project(project_id=d8c317cfa3fe4e0aa6bdd74dbc68a586)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:53.902941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483f279-c280-42f6-b757-f17f621a78ed None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:53.905059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 205/409] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:53 2022] GET /identity/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:53.905936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483f279-c280-42f6-b757-f17f621a78ed None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:53.964430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fda304-65b8-4449-9de4-7ae4248d8b1d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:53.983466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:54.001492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5099f7-ac9e-449b-a704-fb8cddf0e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:54.002268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5099f7-ac9e-449b-a704-fb8cddf0e272 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.006636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5099f7-ac9e-449b-a704-fb8cddf0e272 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.006636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5099f7-ac9e-449b-a704-fb8cddf0e272 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.006636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5099f7-ac9e-449b-a704-fb8cddf0e272 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:54.006636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5099f7-ac9e-449b-a704-fb8cddf0e272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:54.006636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5099f7-ac9e-449b-a704-fb8cddf0e272 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:54.018271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483f279-c280-42f6-b757-f17f621a78ed None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:54.018428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 205/410] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:54.019721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7606b-14a4-4638-945e-7114b9c7f5d5 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.020008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7606b-14a4-4638-945e-7114b9c7f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.020283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7606b-14a4-4638-945e-7114b9c7f5d5 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.021665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 206/411] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:54.022413 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7606b-14a4-4638-945e-7114b9c7f5d5 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:54.072247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec0565-abb6-452b-9448-d2aedcf0a209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:54.073162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec0565-abb6-452b-9448-d2aedcf0a209 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.078359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec0565-abb6-452b-9448-d2aedcf0a209 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.078771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec0565-abb6-452b-9448-d2aedcf0a209 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.081019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec0565-abb6-452b-9448-d2aedcf0a209 None admin] RBAC: Authorizing `identity:create_grant(project_id=488c0084475d48098c54c5f7ee0fe92f, group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:54.082530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec0565-abb6-452b-9448-d2aedcf0a209 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:54.097322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:54.136326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7606b-14a4-4638-945e-7114b9c7f5d5 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:54.169829 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d627ca-d897-486d-8cca-c185f96a10d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:54.170727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d627ca-d897-486d-8cca-c185f96a10d7 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.171013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d627ca-d897-486d-8cca-c185f96a10d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.171335 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d627ca-d897-486d-8cca-c185f96a10d7 None admin] PATH_INFO: `/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586/groups/52b9012085a9417d94d28d7ee872c4ab/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.173280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d627ca-d897-486d-8cca-c185f96a10d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8c317cfa3fe4e0aa6bdd74dbc68a586, group_id=52b9012085a9417d94d28d7ee872c4ab, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:54.174740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d627ca-d897-486d-8cca-c185f96a10d7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:54.184011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 206/412] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 14 15:34:53 2022] PUT /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:54.184881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec0565-abb6-452b-9448-d2aedcf0a209 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:54.247599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 207/413] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 14 15:34:54 2022] PUT /identity/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586/groups/52b9012085a9417d94d28d7ee872c4ab/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:54.255769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9e442-7692-45bf-a284-928205098c10 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.256263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9e442-7692-45bf-a284-928205098c10 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.256742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9e442-7692-45bf-a284-928205098c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.295598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cdd9e442-7692-45bf-a284-928205098c10 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:54.297173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cdd9e442-7692-45bf-a284-928205098c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:54.324622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec0565-abb6-452b-9448-d2aedcf0a209 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:54.365067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:54.373374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f6c65-2d5d-4e1c-88ff-d1e3ed1b73a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:54.379477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f6c65-2d5d-4e1c-88ff-d1e3ed1b73a5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.379477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f6c65-2d5d-4e1c-88ff-d1e3ed1b73a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.379477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f6c65-2d5d-4e1c-88ff-d1e3ed1b73a5 None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.379477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f6c65-2d5d-4e1c-88ff-d1e3ed1b73a5 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:54.380094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f6c65-2d5d-4e1c-88ff-d1e3ed1b73a5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:54.383520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 207/414] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:54 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:54.389733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f6c65-2d5d-4e1c-88ff-d1e3ed1b73a5 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:54.406092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 208/415] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:54.431854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9e442-7692-45bf-a284-928205098c10 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:54.501205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:54.522656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f6c65-2d5d-4e1c-88ff-d1e3ed1b73a5 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:54.556608 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9e442-7692-45bf-a284-928205098c10 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:54.563485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddc39f-cd1c-462c-958f-8db514a67b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:54.564531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddc39f-cd1c-462c-958f-8db514a67b03 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.565628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddc39f-cd1c-462c-958f-8db514a67b03 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.565628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddc39f-cd1c-462c-958f-8db514a67b03 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.567903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddc39f-cd1c-462c-958f-8db514a67b03 None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:54.569458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddc39f-cd1c-462c-958f-8db514a67b03 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:54.571641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 208/416] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:54 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:54.578061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddc39f-cd1c-462c-958f-8db514a67b03 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:54.612161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31f2b6-9487-40ab-9064-054a4ac37f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:54.613034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31f2b6-9487-40ab-9064-054a4ac37f6d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.613317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31f2b6-9487-40ab-9064-054a4ac37f6d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.613592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31f2b6-9487-40ab-9064-054a4ac37f6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.631198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31f2b6-9487-40ab-9064-054a4ac37f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-eb31f2b6-9487-40ab-9064-054a4ac37f6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:54.646361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:54.647302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:54.648109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 209/417] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:54.654859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31f2b6-9487-40ab-9064-054a4ac37f6d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:54.730200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:54.788747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31f2b6-9487-40ab-9064-054a4ac37f6d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:54.792522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddc39f-cd1c-462c-958f-8db514a67b03 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:54.805162 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:54.844996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff7b84-a97c-45e1-a469-6284acf98241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:54.845677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff7b84-a97c-45e1-a469-6284acf98241 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.845992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff7b84-a97c-45e1-a469-6284acf98241 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.846277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff7b84-a97c-45e1-a469-6284acf98241 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.847035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff7b84-a97c-45e1-a469-6284acf98241 None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:54.847344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff7b84-a97c-45e1-a469-6284acf98241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:54.848723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff7b84-a97c-45e1-a469-6284acf98241 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:54.860764 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3459fa-fecd-49c3-bcdc-e73995968820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:54.861664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3459fa-fecd-49c3-bcdc-e73995968820 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:54.861984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3459fa-fecd-49c3-bcdc-e73995968820 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:54.862266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3459fa-fecd-49c3-bcdc-e73995968820 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54/roles/2806b70c57be44f9b260ff5322bf02fb` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:54.864394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 210/418] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:54.864586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3459fa-fecd-49c3-bcdc-e73995968820 None admin] RBAC: Authorizing `identity:create_grant(project_id=488c0084475d48098c54c5f7ee0fe92f, group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role_id=2806b70c57be44f9b260ff5322bf02fb)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:54.869020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3459fa-fecd-49c3-bcdc-e73995968820 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:54.870425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff7b84-a97c-45e1-a469-6284acf98241 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:54.970333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 209/419] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 14 15:34:54 2022] PUT /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54/roles/2806b70c57be44f9b260ff5322bf02fb => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:34:54.997391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff7b84-a97c-45e1-a469-6284acf98241 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:55.039444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c9c0a-6552-469a-b7f6-197200045b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:55.039444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c9c0a-6552-469a-b7f6-197200045b30 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:55.039444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c9c0a-6552-469a-b7f6-197200045b30 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:55.039444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c9c0a-6552-469a-b7f6-197200045b30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:55.081918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ef71-1729-4e31-88d0-5baf7cecdddd None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:55.082331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ef71-1729-4e31-88d0-5baf7cecdddd None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:55.082533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1ef71-1729-4e31-88d0-5baf7cecdddd None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:55.084122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 210/420] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:34:55.089706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4fa9e-c878-4700-938b-b8ef42cca843 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:55.090332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4fa9e-c878-4700-938b-b8ef42cca843 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:55.091376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4fa9e-c878-4700-938b-b8ef42cca843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:55.099789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:55.128275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c9c0a-6552-469a-b7f6-197200045b30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:55.129986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c9c0a-6552-469a-b7f6-197200045b30 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:55.139041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-cfb4fa9e-c878-4700-938b-b8ef42cca843 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:55.140616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-cfb4fa9e-c878-4700-938b-b8ef42cca843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:55.232705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 211/421] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:55.252611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 211/422] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:55.258188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4fa9e-c878-4700-938b-b8ef42cca843 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:55.273846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c9c0a-6552-469a-b7f6-197200045b30 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:55.357299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:55.383423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c9c0a-6552-469a-b7f6-197200045b30 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:55.384977 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:55.424799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7d1a3-34ad-49c8-bc71-ff6d25cfc13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:55.426161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7d1a3-34ad-49c8-bc71-ff6d25cfc13a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:55.426684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7d1a3-34ad-49c8-bc71-ff6d25cfc13a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:55.427182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7d1a3-34ad-49c8-bc71-ff6d25cfc13a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:55.439988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb7d1a3-34ad-49c8-bc71-ff6d25cfc13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-6eb7d1a3-34ad-49c8-bc71-ff6d25cfc13a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:55.449668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:55.451131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:55.452401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:55.455042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 212/423] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:55.462074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7d1a3-34ad-49c8-bc71-ff6d25cfc13a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:55.523611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4fa9e-c878-4700-938b-b8ef42cca843 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:55.595973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b90332-c9a6-4d19-a9e1-13dbc75b6093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:55.596854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b90332-c9a6-4d19-a9e1-13dbc75b6093 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:55.597144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b90332-c9a6-4d19-a9e1-13dbc75b6093 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:55.597420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b90332-c9a6-4d19-a9e1-13dbc75b6093 None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:55.601771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b90332-c9a6-4d19-a9e1-13dbc75b6093 None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:55.603288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b90332-c9a6-4d19-a9e1-13dbc75b6093 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:55.612014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7d1a3-34ad-49c8-bc71-ff6d25cfc13a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:55.613605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 212/424] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:55 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:55.614517 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b90332-c9a6-4d19-a9e1-13dbc75b6093 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:55.681333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ff3eb-f602-419d-9219-d7283720ebd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:55.682047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ff3eb-f602-419d-9219-d7283720ebd7 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:55.682333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ff3eb-f602-419d-9219-d7283720ebd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:55.682610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ff3eb-f602-419d-9219-d7283720ebd7 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:55.683367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ff3eb-f602-419d-9219-d7283720ebd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:55.683677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ff3eb-f602-419d-9219-d7283720ebd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:55.685586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ff3eb-f602-419d-9219-d7283720ebd7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:55.694136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:55.703336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 213/425] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:55.709095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ff3eb-f602-419d-9219-d7283720ebd7 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:55.766645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b90332-c9a6-4d19-a9e1-13dbc75b6093 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:55.793114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:55.803309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d7652-aa7a-4e48-9f5e-8488acd005bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:55.804228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d7652-aa7a-4e48-9f5e-8488acd005bb None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:55.804570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d7652-aa7a-4e48-9f5e-8488acd005bb None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:55.804861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d7652-aa7a-4e48-9f5e-8488acd005bb None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:55.806180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d7652-aa7a-4e48-9f5e-8488acd005bb None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:55.807886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d7652-aa7a-4e48-9f5e-8488acd005bb None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:55.810563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 213/426] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:55 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:55.816622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d7652-aa7a-4e48-9f5e-8488acd005bb None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:55.895906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ff3eb-f602-419d-9219-d7283720ebd7 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:55.941981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d7652-aa7a-4e48-9f5e-8488acd005bb None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:55.978211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7ee5d-604c-4db0-b689-8ccb8a4aa6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:55.978914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7ee5d-604c-4db0-b689-8ccb8a4aa6ed None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:55.979205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7ee5d-604c-4db0-b689-8ccb8a4aa6ed None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:55.979483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7ee5d-604c-4db0-b689-8ccb8a4aa6ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:55.980436 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7ee5d-604c-4db0-b689-8ccb8a4aa6ed None admin] RBAC: Adding query filter params (group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=c89e3b28dc2a41c1822590c6a086be14) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:55.980757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7ee5d-604c-4db0-b689-8ccb8a4aa6ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:55.982315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7ee5d-604c-4db0-b689-8ccb8a4aa6ed None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:55.988835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 214/427] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:55 2022] GET /identity/v3/role_assignments?group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:56.016117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5d2f9-263b-4e04-b875-e32b2c1420a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:56.016804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5d2f9-263b-4e04-b875-e32b2c1420a0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:56.017084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5d2f9-263b-4e04-b875-e32b2c1420a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:56.017358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5d2f9-263b-4e04-b875-e32b2c1420a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:56.140721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:56.144883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5d2f9-263b-4e04-b875-e32b2c1420a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:56.148011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5d2f9-263b-4e04-b875-e32b2c1420a0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:56.255306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 214/428] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:56.260668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7ee5d-604c-4db0-b689-8ccb8a4aa6ed None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:56.338493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:56.408822 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7ee5d-604c-4db0-b689-8ccb8a4aa6ed None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:56.448618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af632b-0116-47ac-9815-e941844fbb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:56.449720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af632b-0116-47ac-9815-e941844fbb0c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:56.450181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af632b-0116-47ac-9815-e941844fbb0c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:56.450533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af632b-0116-47ac-9815-e941844fbb0c None admin] PATH_INFO: `/v3/projects/c89e3b28dc2a41c1822590c6a086be14` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:56.451920 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af632b-0116-47ac-9815-e941844fbb0c None admin] RBAC: Authorizing `identity:get_project(project_id=c89e3b28dc2a41c1822590c6a086be14)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:56.453717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af632b-0116-47ac-9815-e941844fbb0c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:56.455785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 215/429] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:56 2022] GET /identity/v3/projects/c89e3b28dc2a41c1822590c6a086be14 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:56.462408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5d2f9-263b-4e04-b875-e32b2c1420a0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:56.567071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22b4db-1113-4f34-be24-93ba4fbaa392 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:56.567391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22b4db-1113-4f34-be24-93ba4fbaa392 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:56.567626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22b4db-1113-4f34-be24-93ba4fbaa392 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:56.569175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 216/430] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:34:56.574359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c999ba-893d-4406-ae6f-2822818f5553 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:56.574655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c999ba-893d-4406-ae6f-2822818f5553 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:56.574958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c999ba-893d-4406-ae6f-2822818f5553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:56.597397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-66c999ba-893d-4406-ae6f-2822818f5553 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:56.598730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-66c999ba-893d-4406-ae6f-2822818f5553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:56.620216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:56.624206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5d2f9-263b-4e04-b875-e32b2c1420a0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:56.661280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a223aed-e224-4a20-ad72-8e707f652e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:56.662178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a223aed-e224-4a20-ad72-8e707f652e9a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:56.662528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a223aed-e224-4a20-ad72-8e707f652e9a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:56.662816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a223aed-e224-4a20-ad72-8e707f652e9a None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:56.664010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a223aed-e224-4a20-ad72-8e707f652e9a None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:56.665455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a223aed-e224-4a20-ad72-8e707f652e9a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:56.672871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 215/431] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:56 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:56.673830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a223aed-e224-4a20-ad72-8e707f652e9a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:56.683756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 217/432] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:56.706884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c999ba-893d-4406-ae6f-2822818f5553 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:56.778767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:56.781987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a223aed-e224-4a20-ad72-8e707f652e9a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:56.822065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3c339-1198-4021-b691-4ef9a0269535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:56.822841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3c339-1198-4021-b691-4ef9a0269535 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:56.823125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3c339-1198-4021-b691-4ef9a0269535 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:56.823401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3c339-1198-4021-b691-4ef9a0269535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:56.824242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3c339-1198-4021-b691-4ef9a0269535 None admin] RBAC: Adding query filter params (group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=c89e3b28dc2a41c1822590c6a086be14) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:56.824554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3c339-1198-4021-b691-4ef9a0269535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:56.826019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c999ba-893d-4406-ae6f-2822818f5553 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:56.826566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3c339-1198-4021-b691-4ef9a0269535 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:56.833394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 216/433] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:56 2022] GET /identity/v3/role_assignments?group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54&scope.project.id=c89e3b28dc2a41c1822590c6a086be14&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:56.868229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d744e-9489-4fd6-94e4-51e68c1f2449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:56.868976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d744e-9489-4fd6-94e4-51e68c1f2449 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:56.869260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d744e-9489-4fd6-94e4-51e68c1f2449 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:56.869528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d744e-9489-4fd6-94e4-51e68c1f2449 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:56.881688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d744e-9489-4fd6-94e4-51e68c1f2449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-ec5d744e-9489-4fd6-94e4-51e68c1f2449 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:56.890992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:56.892783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:56.894257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 218/434] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:56.899855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3c339-1198-4021-b691-4ef9a0269535 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:56.949164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:57.021620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bd1ed-6465-4420-b4eb-1739d1394087 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.022183 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bd1ed-6465-4420-b4eb-1739d1394087 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.022183 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bd1ed-6465-4420-b4eb-1739d1394087 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.023689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 219/435] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:34:57.029306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443c24d-d94f-4877-817a-98d755178b86 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.029914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443c24d-d94f-4877-817a-98d755178b86 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.030365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443c24d-d94f-4877-817a-98d755178b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.045871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3c339-1198-4021-b691-4ef9a0269535 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:57.062781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-1443c24d-d94f-4877-817a-98d755178b86 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:57.064302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1443c24d-d94f-4877-817a-98d755178b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:57.088843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d1b9e-04ac-426f-af2b-50c897374125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:57.093873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d1b9e-04ac-426f-af2b-50c897374125 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.093873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d1b9e-04ac-426f-af2b-50c897374125 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.093873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d1b9e-04ac-426f-af2b-50c897374125 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.093873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d1b9e-04ac-426f-af2b-50c897374125 None admin] RBAC: Adding query filter params (name=member) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:57.093873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d1b9e-04ac-426f-af2b-50c897374125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:57.093873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d1b9e-04ac-426f-af2b-50c897374125 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:57.105428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 217/436] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:34:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:57.111532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d1b9e-04ac-426f-af2b-50c897374125 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:57.154249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 220/437] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:34:57.173959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443c24d-d94f-4877-817a-98d755178b86 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:57.220185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:57.231016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:57.236076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d1b9e-04ac-426f-af2b-50c897374125 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:57.285091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c20426-0cf7-4a58-b30d-7d111052c7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:57.285748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c20426-0cf7-4a58-b30d-7d111052c7f6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.286086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c20426-0cf7-4a58-b30d-7d111052c7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.286367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c20426-0cf7-4a58-b30d-7d111052c7f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.290628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443c24d-d94f-4877-817a-98d755178b86 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:57.337953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdfaf5-26cc-49bf-8384-3accb3e25040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:57.338769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdfaf5-26cc-49bf-8384-3accb3e25040 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.339090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdfaf5-26cc-49bf-8384-3accb3e25040 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.339617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdfaf5-26cc-49bf-8384-3accb3e25040 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.353273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdfaf5-26cc-49bf-8384-3accb3e25040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-e6cdfaf5-26cc-49bf-8384-3accb3e25040 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:57.364263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:57.365224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:34:57.366070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:34:57.366724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 221/438] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:34:57.367404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c20426-0cf7-4a58-b30d-7d111052c7f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:57.369097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c20426-0cf7-4a58-b30d-7d111052c7f6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:57.374217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdfaf5-26cc-49bf-8384-3accb3e25040 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:57.471970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:57.482591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 218/439] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:57.484231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc628e-c250-4d5e-95ad-ac40b7b2943e None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.484231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc628e-c250-4d5e-95ad-ac40b7b2943e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.484438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc628e-c250-4d5e-95ad-ac40b7b2943e None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.485703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 219/440] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:34:57.487855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc628e-c250-4d5e-95ad-ac40b7b2943e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:57.490843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdfaf5-26cc-49bf-8384-3accb3e25040 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:57.493430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:57.524854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04002f-5586-46f4-97c4-8ffd76945c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:57.525526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04002f-5586-46f4-97c4-8ffd76945c4c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.525842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04002f-5586-46f4-97c4-8ffd76945c4c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.526192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04002f-5586-46f4-97c4-8ffd76945c4c None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.526947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04002f-5586-46f4-97c4-8ffd76945c4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:57.527256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04002f-5586-46f4-97c4-8ffd76945c4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:57.529166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04002f-5586-46f4-97c4-8ffd76945c4c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:57.537595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 222/441] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:57.537595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366967c-f19d-45c2-a6ec-ab765d8b1127 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.537595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366967c-f19d-45c2-a6ec-ab765d8b1127 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.537595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366967c-f19d-45c2-a6ec-ab765d8b1127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.555835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-1366967c-f19d-45c2-a6ec-ab765d8b1127 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:34:57.557025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1366967c-f19d-45c2-a6ec-ab765d8b1127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:34:57.594297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc628e-c250-4d5e-95ad-ac40b7b2943e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:57.623968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 223/442] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:34:57.624885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366967c-f19d-45c2-a6ec-ab765d8b1127 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:57.630100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b875dfd-5b20-4a72-9e2f-f16c60eb63ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:57.631187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b875dfd-5b20-4a72-9e2f-f16c60eb63ff None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.631648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b875dfd-5b20-4a72-9e2f-f16c60eb63ff None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.632170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b875dfd-5b20-4a72-9e2f-f16c60eb63ff None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.633964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b875dfd-5b20-4a72-9e2f-f16c60eb63ff None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:57.636344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b875dfd-5b20-4a72-9e2f-f16c60eb63ff None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:57.638371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 220/443] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:57 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:57.643625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b875dfd-5b20-4a72-9e2f-f16c60eb63ff None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:57.702477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:57.722145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:57.727855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366967c-f19d-45c2-a6ec-ab765d8b1127 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:57.741808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b875dfd-5b20-4a72-9e2f-f16c60eb63ff None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:57.762296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781897b-d1b1-40d9-b297-a2cb010ca2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:57.762984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781897b-d1b1-40d9-b297-a2cb010ca2f0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.763275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781897b-d1b1-40d9-b297-a2cb010ca2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.763546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781897b-d1b1-40d9-b297-a2cb010ca2f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.775316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd629d1-7592-49b9-9c30-2e1c98c81c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:57.776168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd629d1-7592-49b9-9c30-2e1c98c81c7e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.776453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd629d1-7592-49b9-9c30-2e1c98c81c7e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.776729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd629d1-7592-49b9-9c30-2e1c98c81c7e None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.777991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd629d1-7592-49b9-9c30-2e1c98c81c7e None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:57.779490 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd629d1-7592-49b9-9c30-2e1c98c81c7e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:57.781112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 221/444] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:57 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:57.781888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd629d1-7592-49b9-9c30-2e1c98c81c7e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:57.832488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781897b-d1b1-40d9-b297-a2cb010ca2f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:57.833991 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781897b-d1b1-40d9-b297-a2cb010ca2f0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:57.878752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd629d1-7592-49b9-9c30-2e1c98c81c7e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:57.905246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:57.912661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544494a-21af-4a0c-bd44-8a01918eeae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:57.913468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544494a-21af-4a0c-bd44-8a01918eeae8 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:57.913752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544494a-21af-4a0c-bd44-8a01918eeae8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:57.914077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544494a-21af-4a0c-bd44-8a01918eeae8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:57.924467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544494a-21af-4a0c-bd44-8a01918eeae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:34:57.925890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:57.932241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 224/445] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-b544494a-21af-4a0c-bd44-8a01918eeae8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:34:57.932841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:34:57.934166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:34:57.935687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:34:57.936449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781897b-d1b1-40d9-b297-a2cb010ca2f0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:57.938477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 222/446] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:34:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:34:57.939298 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544494a-21af-4a0c-bd44-8a01918eeae8 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:58.038415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544494a-21af-4a0c-bd44-8a01918eeae8 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:58.039082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781897b-d1b1-40d9-b297-a2cb010ca2f0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:58.072961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce23bb1-fcc3-4c5a-a99a-eab540a45312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:58.073884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce23bb1-fcc3-4c5a-a99a-eab540a45312 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:58.074165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce23bb1-fcc3-4c5a-a99a-eab540a45312 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:58.074434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce23bb1-fcc3-4c5a-a99a-eab540a45312 None admin] PATH_INFO: `/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:58.074920 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412dab6-95a8-428c-8a90-8a3907464e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:58.075613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412dab6-95a8-428c-8a90-8a3907464e66 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:58.075717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce23bb1-fcc3-4c5a-a99a-eab540a45312 None admin] RBAC: Authorizing `identity:get_project(project_id=d8c317cfa3fe4e0aa6bdd74dbc68a586)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:58.075898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412dab6-95a8-428c-8a90-8a3907464e66 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:58.076181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412dab6-95a8-428c-8a90-8a3907464e66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:58.077013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412dab6-95a8-428c-8a90-8a3907464e66 None admin] RBAC: Adding query filter params (group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role.id=fda37aea043b4685b930f1d9bde2b0ec, scope.project.id=488c0084475d48098c54c5f7ee0fe92f) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:58.077101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce23bb1-fcc3-4c5a-a99a-eab540a45312 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:58.077327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412dab6-95a8-428c-8a90-8a3907464e66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:58.078775 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 223/447] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:57 2022] GET /identity/v3/projects/d8c317cfa3fe4e0aa6bdd74dbc68a586 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:58.078900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412dab6-95a8-428c-8a90-8a3907464e66 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:58.079529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce23bb1-fcc3-4c5a-a99a-eab540a45312 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:58.098487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 225/448] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:57 2022] GET /identity/v3/role_assignments?group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=fda37aea043b4685b930f1d9bde2b0ec => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:58.098487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412dab6-95a8-428c-8a90-8a3907464e66 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:58.158222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:58.160470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:58.203830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412dab6-95a8-428c-8a90-8a3907464e66 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:58.211356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce23bb1-fcc3-4c5a-a99a-eab540a45312 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:58.243895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96525304-dd40-4664-96b2-f0921dd48ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:58.244749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96525304-dd40-4664-96b2-f0921dd48ce0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:58.245033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96525304-dd40-4664-96b2-f0921dd48ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:58.245306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96525304-dd40-4664-96b2-f0921dd48ce0 None admin] PATH_INFO: `/v3/groups/52b9012085a9417d94d28d7ee872c4ab` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:58.246547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96525304-dd40-4664-96b2-f0921dd48ce0 None admin] RBAC: Authorizing `identity:get_group(group_id=52b9012085a9417d94d28d7ee872c4ab)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:58.248043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96525304-dd40-4664-96b2-f0921dd48ce0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:58.249650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 226/449] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:58 2022] GET /identity/v3/groups/52b9012085a9417d94d28d7ee872c4ab => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:58.250884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a991c7-ca74-4b26-8062-3eadb0387332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:58.251560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a991c7-ca74-4b26-8062-3eadb0387332 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:58.251839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a991c7-ca74-4b26-8062-3eadb0387332 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:58.252114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a991c7-ca74-4b26-8062-3eadb0387332 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:58.252863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a991c7-ca74-4b26-8062-3eadb0387332 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:58.253166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a991c7-ca74-4b26-8062-3eadb0387332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:58.255477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96525304-dd40-4664-96b2-f0921dd48ce0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:58.255861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a991c7-ca74-4b26-8062-3eadb0387332 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:58.264919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 224/450] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:34:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:58.270265 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a991c7-ca74-4b26-8062-3eadb0387332 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:58.376237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96525304-dd40-4664-96b2-f0921dd48ce0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:58.384908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a991c7-ca74-4b26-8062-3eadb0387332 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:58.407477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:58.413129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:58.422552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2e458-caad-4096-827b-745f20464b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:58.423218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2e458-caad-4096-827b-745f20464b9c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:58.423505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2e458-caad-4096-827b-745f20464b9c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:58.423831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2e458-caad-4096-827b-745f20464b9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:58.424630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2e458-caad-4096-827b-745f20464b9c None admin] RBAC: Adding query filter params (group.id=52b9012085a9417d94d28d7ee872c4ab, role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=d8c317cfa3fe4e0aa6bdd74dbc68a586) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:58.424949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2e458-caad-4096-827b-745f20464b9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:58.426621 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2e458-caad-4096-827b-745f20464b9c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:58.427912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e315ef-162c-4da7-8684-96b37a3b1751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:58.429240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e315ef-162c-4da7-8684-96b37a3b1751 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:58.429633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e315ef-162c-4da7-8684-96b37a3b1751 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:58.433480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e315ef-162c-4da7-8684-96b37a3b1751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:58.434368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 227/451] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:58 2022] GET /identity/v3/role_assignments?group.id=52b9012085a9417d94d28d7ee872c4ab&scope.project.id=d8c317cfa3fe4e0aa6bdd74dbc68a586&role.id=abfb60dccca7470295258595a66a5d17 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:34:58.507642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e315ef-162c-4da7-8684-96b37a3b1751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:58.509144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e315ef-162c-4da7-8684-96b37a3b1751 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:58.633275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 225/452] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:34:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 15:34:58.639501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2e458-caad-4096-827b-745f20464b9c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:58.741342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2e458-caad-4096-827b-745f20464b9c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:58.775134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121954f-af28-4001-aa48-bd520c3b6347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:58.775984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121954f-af28-4001-aa48-bd520c3b6347 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:58.776270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121954f-af28-4001-aa48-bd520c3b6347 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:58.776545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121954f-af28-4001-aa48-bd520c3b6347 None admin] PATH_INFO: `/v3/projects/488c0084475d48098c54c5f7ee0fe92f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:58.777717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121954f-af28-4001-aa48-bd520c3b6347 None admin] RBAC: Authorizing `identity:get_project(project_id=488c0084475d48098c54c5f7ee0fe92f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:58.779218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121954f-af28-4001-aa48-bd520c3b6347 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:58.780900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 228/453] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:34:58 2022] GET /identity/v3/projects/488c0084475d48098c54c5f7ee0fe92f => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:58.786532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e315ef-162c-4da7-8684-96b37a3b1751 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:58.804300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:58.881921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e315ef-162c-4da7-8684-96b37a3b1751 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:58.915383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5bc42-c2bb-4190-8df5-14e35e569b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:58.916175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5bc42-c2bb-4190-8df5-14e35e569b51 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:58.916455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5bc42-c2bb-4190-8df5-14e35e569b51 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:58.916738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5bc42-c2bb-4190-8df5-14e35e569b51 None admin] PATH_INFO: `/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:58.917910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5bc42-c2bb-4190-8df5-14e35e569b51 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a2fe9a499a4b9ebe92cc3cd9b58e54)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:58.919356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5bc42-c2bb-4190-8df5-14e35e569b51 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:58.920938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 226/454] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 14 15:34:58 2022] GET /identity/v3/groups/b0a2fe9a499a4b9ebe92cc3cd9b58e54 => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:34:58.926662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121954f-af28-4001-aa48-bd520c3b6347 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:34:58.985263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:34:59.026120 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121954f-af28-4001-aa48-bd520c3b6347 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:34:59.060095 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4189df-4bbb-442d-ab10-6d7e500ae7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:34:59.060752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4189df-4bbb-442d-ab10-6d7e500ae7e3 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:34:59.061038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4189df-4bbb-442d-ab10-6d7e500ae7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:34:59.061358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4189df-4bbb-442d-ab10-6d7e500ae7e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:34:59.062251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4189df-4bbb-442d-ab10-6d7e500ae7e3 None admin] RBAC: Adding query filter params (group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54, role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=488c0084475d48098c54c5f7ee0fe92f) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:34:59.062564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4189df-4bbb-442d-ab10-6d7e500ae7e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:34:59.064045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4189df-4bbb-442d-ab10-6d7e500ae7e3 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:34:59.070123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 229/455] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 14 15:34:58 2022] GET /identity/v3/role_assignments?group.id=b0a2fe9a499a4b9ebe92cc3cd9b58e54&scope.project.id=488c0084475d48098c54c5f7ee0fe92f&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:01.647227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bedf1-c1b9-40d9-a1fe-90e2111bd232 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:01.648257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bedf1-c1b9-40d9-a1fe-90e2111bd232 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:01.648742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bedf1-c1b9-40d9-a1fe-90e2111bd232 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:01.650647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 227/456] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:01.655990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523cace-0849-467a-874b-9c0d1e3ae047 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:01.656796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523cace-0849-467a-874b-9c0d1e3ae047 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:01.657286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523cace-0849-467a-874b-9c0d1e3ae047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:01.682819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-4523cace-0849-467a-874b-9c0d1e3ae047 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:01.684891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4523cace-0849-467a-874b-9c0d1e3ae047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:01.779148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 230/457] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:01.816070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bedf1-c1b9-40d9-a1fe-90e2111bd232 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:01.930525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:01.948719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42de787-734a-40e3-b7ee-b483accf40da None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:01.949527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42de787-734a-40e3-b7ee-b483accf40da None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:01.950239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42de787-734a-40e3-b7ee-b483accf40da None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:01.950756 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bedf1-c1b9-40d9-a1fe-90e2111bd232 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:01.953308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 231/458] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:01.957739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837361c-e383-4432-acc5-b20c078f3880 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:01.958309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837361c-e383-4432-acc5-b20c078f3880 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:01.958828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837361c-e383-4432-acc5-b20c078f3880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:01.979401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-2837361c-e383-4432-acc5-b20c078f3880 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:01.983937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2837361c-e383-4432-acc5-b20c078f3880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:02.008591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfb163-f132-4284-b448-68dc5305d259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:02.009992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfb163-f132-4284-b448-68dc5305d259 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.011920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfb163-f132-4284-b448-68dc5305d259 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.012464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfb163-f132-4284-b448-68dc5305d259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.032594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfb163-f132-4284-b448-68dc5305d259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:02.035223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfb163-f132-4284-b448-68dc5305d259 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-20dfb163-f132-4284-b448-68dc5305d259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:02.068531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:02.069474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:02.070368 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:02.070368 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:02.070368 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:02.070368 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:02.082035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 228/459] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:02.087785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfb163-f132-4284-b448-68dc5305d259 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:02.095687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:02.122534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 232/460] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:35:02.122534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056671b-f02f-4eef-b9d9-ef42bff0d484 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.122534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056671b-f02f-4eef-b9d9-ef42bff0d484 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.122534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056671b-f02f-4eef-b9d9-ef42bff0d484 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.122534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 233/461] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:02.127896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386f12d-a00f-4ba1-8fe0-9ca832a48418 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.128189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386f12d-a00f-4ba1-8fe0-9ca832a48418 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.128473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386f12d-a00f-4ba1-8fe0-9ca832a48418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.155528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6386f12d-a00f-4ba1-8fe0-9ca832a48418 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:02.157039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6386f12d-a00f-4ba1-8fe0-9ca832a48418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:02.226966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfb163-f132-4284-b448-68dc5305d259 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:02.229403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 234/462] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:02.230566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386f12d-a00f-4ba1-8fe0-9ca832a48418 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:02.279902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7548ae-4423-45fc-8f48-dadda53e2c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:02.280817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7548ae-4423-45fc-8f48-dadda53e2c4e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.280817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7548ae-4423-45fc-8f48-dadda53e2c4e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.281127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7548ae-4423-45fc-8f48-dadda53e2c4e None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.282029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7548ae-4423-45fc-8f48-dadda53e2c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:02.282342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7548ae-4423-45fc-8f48-dadda53e2c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:02.283709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7548ae-4423-45fc-8f48-dadda53e2c4e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:02.294424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:02.296940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 229/463] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:02.301295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7548ae-4423-45fc-8f48-dadda53e2c4e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:02.349884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386f12d-a00f-4ba1-8fe0-9ca832a48418 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:02.366325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:02.387381 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024bba6-2b15-413f-9ac0-66dee8ae83bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:02.388545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024bba6-2b15-413f-9ac0-66dee8ae83bc None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.389035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024bba6-2b15-413f-9ac0-66dee8ae83bc None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.389313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024bba6-2b15-413f-9ac0-66dee8ae83bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.399789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024bba6-2b15-413f-9ac0-66dee8ae83bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:02.401632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024bba6-2b15-413f-9ac0-66dee8ae83bc None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-b024bba6-2b15-413f-9ac0-66dee8ae83bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:02.421758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:02.422684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:02.423580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:02.423580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:02.423580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:02.423580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:02.423791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7548ae-4423-45fc-8f48-dadda53e2c4e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:02.426119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 235/464] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:02.426783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024bba6-2b15-413f-9ac0-66dee8ae83bc None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:02.458748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691a652-e06a-417d-9ccf-81115c21a44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:02.459546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691a652-e06a-417d-9ccf-81115c21a44a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.459830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691a652-e06a-417d-9ccf-81115c21a44a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.460108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691a652-e06a-417d-9ccf-81115c21a44a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.471868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691a652-e06a-417d-9ccf-81115c21a44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:02.473347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691a652-e06a-417d-9ccf-81115c21a44a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-d691a652-e06a-417d-9ccf-81115c21a44a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:02.491779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:02.492677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:02.493525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:02.493525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:02.493525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:02.493525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:02.493525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 230/465] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:02.494555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f5307-7bd6-4dcd-9ae8-76454feebbc4 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.494783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f5307-7bd6-4dcd-9ae8-76454feebbc4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.495070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f5307-7bd6-4dcd-9ae8-76454feebbc4 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.497943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 231/466] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:02.497943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f5307-7bd6-4dcd-9ae8-76454feebbc4 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:02.547939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024bba6-2b15-413f-9ac0-66dee8ae83bc None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:02.552599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:02.585237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368a098-2467-452f-b14e-3de062fd8d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:02.585950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368a098-2467-452f-b14e-3de062fd8d35 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.586237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368a098-2467-452f-b14e-3de062fd8d35 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.586516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368a098-2467-452f-b14e-3de062fd8d35 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.587233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368a098-2467-452f-b14e-3de062fd8d35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:02.588684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368a098-2467-452f-b14e-3de062fd8d35 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:02.609196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f5307-7bd6-4dcd-9ae8-76454feebbc4 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:02.643420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285189c-449e-4f9f-9b2e-d9de3898ba32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:02.644089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285189c-449e-4f9f-9b2e-d9de3898ba32 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.644372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285189c-449e-4f9f-9b2e-d9de3898ba32 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.644649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285189c-449e-4f9f-9b2e-d9de3898ba32 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.645435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285189c-449e-4f9f-9b2e-d9de3898ba32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:02.645679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285189c-449e-4f9f-9b2e-d9de3898ba32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:02.647077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285189c-449e-4f9f-9b2e-d9de3898ba32 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:02.659367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 232/467] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:02.660163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285189c-449e-4f9f-9b2e-d9de3898ba32 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:02.677786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 236/468] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:35:02 2022] POST /identity/v3/users => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:02.691702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def05e6-9bd4-4110-9d63-9d574441954e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.692008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def05e6-9bd4-4110-9d63-9d574441954e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.692267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def05e6-9bd4-4110-9d63-9d574441954e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.732473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-7def05e6-9bd4-4110-9d63-9d574441954e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:02.734423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7def05e6-9bd4-4110-9d63-9d574441954e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:02.760195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:02.782500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285189c-449e-4f9f-9b2e-d9de3898ba32 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:02.797636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:02.833046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7617b21-80b3-4052-88c4-b6b5a3531ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:02.834095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7617b21-80b3-4052-88c4-b6b5a3531ed4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:02.834799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7617b21-80b3-4052-88c4-b6b5a3531ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:02.835383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7617b21-80b3-4052-88c4-b6b5a3531ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:02.836018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7617b21-80b3-4052-88c4-b6b5a3531ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:02.836429 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7617b21-80b3-4052-88c4-b6b5a3531ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:02.837949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7617b21-80b3-4052-88c4-b6b5a3531ed4 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:02.842433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 237/469] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:02.849837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def05e6-9bd4-4110-9d63-9d574441954e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:02.858151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 233/470] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:02.863894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7617b21-80b3-4052-88c4-b6b5a3531ed4 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:02.990026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def05e6-9bd4-4110-9d63-9d574441954e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:02.991295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7617b21-80b3-4052-88c4-b6b5a3531ed4 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:03.029862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bff883-a04c-442c-892d-fee3fbda099c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:03.029862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bff883-a04c-442c-892d-fee3fbda099c None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:03.030107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bff883-a04c-442c-892d-fee3fbda099c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:03.030201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace33f92-e082-4587-8a80-4c09e89ca613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:03.030479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bff883-a04c-442c-892d-fee3fbda099c None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:03.030889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace33f92-e082-4587-8a80-4c09e89ca613 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:03.031083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bff883-a04c-442c-892d-fee3fbda099c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:03.031179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace33f92-e082-4587-8a80-4c09e89ca613 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:03.031431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace33f92-e082-4587-8a80-4c09e89ca613 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:03.032131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace33f92-e082-4587-8a80-4c09e89ca613 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:03.032519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bff883-a04c-442c-892d-fee3fbda099c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:03.033561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace33f92-e082-4587-8a80-4c09e89ca613 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:03.240841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 238/471] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:35:02 2022] POST /identity/v3/users => generated 248 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:03.241684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace33f92-e082-4587-8a80-4c09e89ca613 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:03.263514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:03.264135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 234/472] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:35:02 2022] POST /identity/v3/users => generated 248 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:03.266756 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7025f6a-f53d-48c6-b6ea-1537154373ab None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:03.267582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7025f6a-f53d-48c6-b6ea-1537154373ab None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:03.268075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7025f6a-f53d-48c6-b6ea-1537154373ab None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:03.279116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 235/473] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:03 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:03.284542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc7f21-5fef-425e-a368-0791f1750c8c None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:03.284832 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc7f21-5fef-425e-a368-0791f1750c8c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:03.285017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc7f21-5fef-425e-a368-0791f1750c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:03.318374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-19bc7f21-5fef-425e-a368-0791f1750c8c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:03.319686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-19bc7f21-5fef-425e-a368-0791f1750c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:03.372397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:03.452955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 236/474] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:03.464556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace33f92-e082-4587-8a80-4c09e89ca613 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:03.477201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc7f21-5fef-425e-a368-0791f1750c8c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:03.508707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213ff18-ae8b-45e8-bd58-f15c3b1e9b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:03.510086 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213ff18-ae8b-45e8-bd58-f15c3b1e9b4c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:03.510686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213ff18-ae8b-45e8-bd58-f15c3b1e9b4c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:03.511163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213ff18-ae8b-45e8-bd58-f15c3b1e9b4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:03.525391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213ff18-ae8b-45e8-bd58-f15c3b1e9b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:03.527570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213ff18-ae8b-45e8-bd58-f15c3b1e9b4c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-3213ff18-ae8b-45e8-bd58-f15c3b1e9b4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:03.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:03.553068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:03.554232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:03.554232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:03.554232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:03.554232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:03.554232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 239/475] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:03.559641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213ff18-ae8b-45e8-bd58-f15c3b1e9b4c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:03.598054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc7f21-5fef-425e-a368-0791f1750c8c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:03.623876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:03.636717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:03.637684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:03.637684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:03.638004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:03.638771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:03.640170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:03.673591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213ff18-ae8b-45e8-bd58-f15c3b1e9b4c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:03.675156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.sql.core [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 15:35:03.675156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 15:35:03.675156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [parameters: {'id': 'ea629495925346f281596fbaee22600a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 15:35:03.675156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92134) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application result.read() Jul 14 15:35:03.708132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:35:03.710268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application yield resource Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise self.value Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application session.commit() Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 15:35:03.712525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.session.flush() Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application compat.raise_( Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 15:35:03.718178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:35:03.720541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application result.read() Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:35:03.722536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [parameters: {'id': 'ea629495925346f281596fbaee22600a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:03.724539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 15:35:03.734156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:03.736202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 237/476] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 14 15:35:03 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:03.736202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:03.737776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc49a5-d125-41fd-9e2d-9ce53cadf94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:03.738539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc49a5-d125-41fd-9e2d-9ce53cadf94b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:03.738829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc49a5-d125-41fd-9e2d-9ce53cadf94b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:03.739099 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc49a5-d125-41fd-9e2d-9ce53cadf94b None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:03.739836 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc49a5-d125-41fd-9e2d-9ce53cadf94b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:03.740140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc49a5-d125-41fd-9e2d-9ce53cadf94b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:03.741532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc49a5-d125-41fd-9e2d-9ce53cadf94b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:03.764100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:03.773854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 240/477] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:03.783480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc49a5-d125-41fd-9e2d-9ce53cadf94b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:03.924882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9a909-83b8-4793-825f-88d3a98ca7b2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:03.945843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc49a5-d125-41fd-9e2d-9ce53cadf94b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:03.973080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:04.002184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41053d9-9c60-4b88-bb72-4b141272ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:04.002874 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41053d9-9c60-4b88-bb72-4b141272ac27 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:04.003162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41053d9-9c60-4b88-bb72-4b141272ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:04.003437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41053d9-9c60-4b88-bb72-4b141272ac27 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:04.004166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41053d9-9c60-4b88-bb72-4b141272ac27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:04.005638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41053d9-9c60-4b88-bb72-4b141272ac27 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:04.008604 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f52e7a-1425-4059-a296-26e788c2ec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:04.009443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f52e7a-1425-4059-a296-26e788c2ec31 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:04.009729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f52e7a-1425-4059-a296-26e788c2ec31 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:04.010029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f52e7a-1425-4059-a296-26e788c2ec31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:04.029927 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f52e7a-1425-4059-a296-26e788c2ec31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-b2f52e7a-1425-4059-a296-26e788c2ec31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:04.039007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:04.040092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 15:35:04.041105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:04.042710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 238/478] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:04.048664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f52e7a-1425-4059-a296-26e788c2ec31 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:04.141556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:04.160627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f52e7a-1425-4059-a296-26e788c2ec31 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:04.194040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 241/479] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:35:03 2022] POST /identity/v3/users => generated 249 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:35:04.206765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf8f60-718d-4547-aba1-f429618a5a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:04.207869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf8f60-718d-4547-aba1-f429618a5a43 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:04.208163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf8f60-718d-4547-aba1-f429618a5a43 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:04.208442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf8f60-718d-4547-aba1-f429618a5a43 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:04.209225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf8f60-718d-4547-aba1-f429618a5a43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:04.209536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf8f60-718d-4547-aba1-f429618a5a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:04.211100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf8f60-718d-4547-aba1-f429618a5a43 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:04.227038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 239/480] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:05.099544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb436e7-5519-4dcf-87d4-840052dce2ba None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:05.100048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb436e7-5519-4dcf-87d4-840052dce2ba None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:05.100123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb436e7-5519-4dcf-87d4-840052dce2ba None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:05.101490 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 242/481] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:05.107015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a0ecb-c1b8-4fa6-9af9-1edccd2ed86f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:05.107826 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a0ecb-c1b8-4fa6-9af9-1edccd2ed86f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:05.108307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a0ecb-c1b8-4fa6-9af9-1edccd2ed86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:05.130638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-427a0ecb-c1b8-4fa6-9af9-1edccd2ed86f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:05.132764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-427a0ecb-c1b8-4fa6-9af9-1edccd2ed86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:05.221118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 240/482] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:05.267364 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb436e7-5519-4dcf-87d4-840052dce2ba None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:05.410580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb436e7-5519-4dcf-87d4-840052dce2ba None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:05.461094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046b7e7-4e57-42a6-b72f-6d4c67ced75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:05.461935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b7e7-4e57-42a6-b72f-6d4c67ced75e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:05.462226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b7e7-4e57-42a6-b72f-6d4c67ced75e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:05.462505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046b7e7-4e57-42a6-b72f-6d4c67ced75e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:05.473626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046b7e7-4e57-42a6-b72f-6d4c67ced75e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-1046b7e7-4e57-42a6-b72f-6d4c67ced75e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:05.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:05.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:05.483197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 243/483] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:05.488588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a0ecb-c1b8-4fa6-9af9-1edccd2ed86f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:05.612406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:05.642280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a0ecb-c1b8-4fa6-9af9-1edccd2ed86f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:05.686118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708f447-11a2-4896-9315-51115c0f13e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:05.686725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708f447-11a2-4896-9315-51115c0f13e2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:05.687008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708f447-11a2-4896-9315-51115c0f13e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:05.687287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708f447-11a2-4896-9315-51115c0f13e2 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:05.688046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708f447-11a2-4896-9315-51115c0f13e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:05.688352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708f447-11a2-4896-9315-51115c0f13e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:05.689723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708f447-11a2-4896-9315-51115c0f13e2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:05.704255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 241/484] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:05.709645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046b7e7-4e57-42a6-b72f-6d4c67ced75e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:05.735117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e6e4e-9d3b-4e38-9fce-96f8ed1e3df8 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:05.735447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e6e4e-9d3b-4e38-9fce-96f8ed1e3df8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:05.735682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e6e4e-9d3b-4e38-9fce-96f8ed1e3df8 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:05.737316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 242/485] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:05.743808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204c78-bc69-4a10-8e69-880d325d6708 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:05.744117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204c78-bc69-4a10-8e69-880d325d6708 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:05.744365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4204c78-bc69-4a10-8e69-880d325d6708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:05.781678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:05.782367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c4204c78-bc69-4a10-8e69-880d325d6708 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:05.783601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c4204c78-bc69-4a10-8e69-880d325d6708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:05.895091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046b7e7-4e57-42a6-b72f-6d4c67ced75e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:05.919756 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 243/486] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:35:05.946914 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4204c78-bc69-4a10-8e69-880d325d6708 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:05.955886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25646b-3b06-430f-bbb4-24b217858957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:05.956705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25646b-3b06-430f-bbb4-24b217858957 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:05.956988 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25646b-3b06-430f-bbb4-24b217858957 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:05.957265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25646b-3b06-430f-bbb4-24b217858957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:05.980371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25646b-3b06-430f-bbb4-24b217858957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:05.983730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25646b-3b06-430f-bbb4-24b217858957 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-ca25646b-3b06-430f-bbb4-24b217858957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:06.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:06.017031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:06.017925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:06.017925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:06.017925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:06.017925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:06.017925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 244/487] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:06.023164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25646b-3b06-430f-bbb4-24b217858957 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:06.055715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:06.098673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4204c78-bc69-4a10-8e69-880d325d6708 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:06.141320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0056dc-53ce-4360-be2c-a74be8306f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:06.142331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0056dc-53ce-4360-be2c-a74be8306f2a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.142690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0056dc-53ce-4360-be2c-a74be8306f2a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.143067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0056dc-53ce-4360-be2c-a74be8306f2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.145992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:06.156465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0056dc-53ce-4360-be2c-a74be8306f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:06.162410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25646b-3b06-430f-bbb4-24b217858957 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-ef0056dc-53ce-4360-be2c-a74be8306f2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:06.164440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:06.165328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:06.166193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:06.171034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 244/488] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:06.172556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cd716-4558-4489-8cf3-09c4a17b97b2 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.172917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cd716-4558-4489-8cf3-09c4a17b97b2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.173558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cd716-4558-4489-8cf3-09c4a17b97b2 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.178868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 245/489] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:06.178868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cd716-4558-4489-8cf3-09c4a17b97b2 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:06.214777 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2fae4-69bb-400b-a805-f2206aa4f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:06.216303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2fae4-69bb-400b-a805-f2206aa4f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.217040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2fae4-69bb-400b-a805-f2206aa4f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.217497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2fae4-69bb-400b-a805-f2206aa4f5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.219282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2fae4-69bb-400b-a805-f2206aa4f5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:06.219803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2fae4-69bb-400b-a805-f2206aa4f5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:06.221374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2fae4-69bb-400b-a805-f2206aa4f5e4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:06.238257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 245/490] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:06.240346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37cd38-5dec-414e-a151-4f9920102631 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.241235 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37cd38-5dec-414e-a151-4f9920102631 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.241694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37cd38-5dec-414e-a151-4f9920102631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.264264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-aa37cd38-5dec-414e-a151-4f9920102631 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:06.265853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-aa37cd38-5dec-414e-a151-4f9920102631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:06.292644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cd716-4558-4489-8cf3-09c4a17b97b2 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:06.319522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:06.332369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec38629-c683-4d16-84c1-6b3a75322099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:06.333302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec38629-c683-4d16-84c1-6b3a75322099 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.333736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec38629-c683-4d16-84c1-6b3a75322099 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.334184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec38629-c683-4d16-84c1-6b3a75322099 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.335085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec38629-c683-4d16-84c1-6b3a75322099 None admin] RBAC: Adding query filter params (name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:06.335541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec38629-c683-4d16-84c1-6b3a75322099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:06.336962 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 246/491] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:06.337264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec38629-c683-4d16-84c1-6b3a75322099 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:06.344730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37cd38-5dec-414e-a151-4f9920102631 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:06.353968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 246/492] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:06.355076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec38629-c683-4d16-84c1-6b3a75322099 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:06.469378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec38629-c683-4d16-84c1-6b3a75322099 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:06.469835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:06.500669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37cd38-5dec-414e-a151-4f9920102631 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:06.511163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192710f-801b-4bd6-bfd9-fd53ba61b45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:06.513011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192710f-801b-4bd6-bfd9-fd53ba61b45f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.513501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192710f-801b-4bd6-bfd9-fd53ba61b45f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.514070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192710f-801b-4bd6-bfd9-fd53ba61b45f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.526394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192710f-801b-4bd6-bfd9-fd53ba61b45f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:06.528138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192710f-801b-4bd6-bfd9-fd53ba61b45f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:06.541809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f543670-1aff-4c15-9ee4-72c3f7f3de0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:06.542626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f543670-1aff-4c15-9ee4-72c3f7f3de0b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.542908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f543670-1aff-4c15-9ee4-72c3f7f3de0b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.543180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f543670-1aff-4c15-9ee4-72c3f7f3de0b None admin] PATH_INFO: `/v3/users/nova` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-7192710f-801b-4bd6-bfd9-fd53ba61b45f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:06.548946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:06.549787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:06.550669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:06.550669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:06.550669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:06.550669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:06.551322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f543670-1aff-4c15-9ee4-72c3f7f3de0b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:06.552875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f543670-1aff-4c15-9ee4-72c3f7f3de0b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:06.557630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 247/493] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:06.558826 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192710f-801b-4bd6-bfd9-fd53ba61b45f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-3f543670-1aff-4c15-9ee4-72c3f7f3de0b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:06.571550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:06.572503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:06.573410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 247/494] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 14 15:35:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:06.574022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f543670-1aff-4c15-9ee4-72c3f7f3de0b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:06.607099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:06.677072 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192710f-801b-4bd6-bfd9-fd53ba61b45f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:06.701805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f543670-1aff-4c15-9ee4-72c3f7f3de0b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:06.716918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a07d6-2269-4e9d-96b9-691d89f8aefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:06.718056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a07d6-2269-4e9d-96b9-691d89f8aefc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.718597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a07d6-2269-4e9d-96b9-691d89f8aefc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.719186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a07d6-2269-4e9d-96b9-691d89f8aefc None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.731255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a07d6-2269-4e9d-96b9-691d89f8aefc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-390a07d6-2269-4e9d-96b9-691d89f8aefc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:06.739899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:06.741005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:06.742058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:06.744019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e5314-164d-4391-8b1f-3daa7df598d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:06.744790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e5314-164d-4391-8b1f-3daa7df598d8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.745184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e5314-164d-4391-8b1f-3daa7df598d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.745551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e5314-164d-4391-8b1f-3daa7df598d8 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.745969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 248/495] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:06.746513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e5314-164d-4391-8b1f-3daa7df598d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:06.746976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e5314-164d-4391-8b1f-3daa7df598d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:06.747410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a07d6-2269-4e9d-96b9-691d89f8aefc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:06.748597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e5314-164d-4391-8b1f-3daa7df598d8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:06.756192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:06.762121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 248/496] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:06.764030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997aea9-1ed6-4302-93ac-5ac1316fe24b None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.771707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997aea9-1ed6-4302-93ac-5ac1316fe24b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.771707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997aea9-1ed6-4302-93ac-5ac1316fe24b None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.771707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 249/497] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:06.771707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997aea9-1ed6-4302-93ac-5ac1316fe24b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:06.851424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:06.855274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a07d6-2269-4e9d-96b9-691d89f8aefc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:06.869136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997aea9-1ed6-4302-93ac-5ac1316fe24b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:06.888899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63fe90-0a59-47f3-8936-d1c0348a978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:06.889925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63fe90-0a59-47f3-8936-d1c0348a978c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.890518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63fe90-0a59-47f3-8936-d1c0348a978c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.890797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63fe90-0a59-47f3-8936-d1c0348a978c None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.891638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63fe90-0a59-47f3-8936-d1c0348a978c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:06.891944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63fe90-0a59-47f3-8936-d1c0348a978c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:06.893379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63fe90-0a59-47f3-8936-d1c0348a978c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:06.902310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689656f-7a0a-48f9-85f3-d0896f80f30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:06.903502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689656f-7a0a-48f9-85f3-d0896f80f30a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.903793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689656f-7a0a-48f9-85f3-d0896f80f30a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.904068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689656f-7a0a-48f9-85f3-d0896f80f30a None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.904812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689656f-7a0a-48f9-85f3-d0896f80f30a None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:06.905118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689656f-7a0a-48f9-85f3-d0896f80f30a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:06.906498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689656f-7a0a-48f9-85f3-d0896f80f30a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:06.914427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 250/498] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:06.915638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae455d-b7b3-4508-af6a-565cff3343dc None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.915926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae455d-b7b3-4508-af6a-565cff3343dc None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.916202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae455d-b7b3-4508-af6a-565cff3343dc None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.917610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 251/499] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:06.920369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae455d-b7b3-4508-af6a-565cff3343dc None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:06.931779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 249/500] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 14 15:35:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:06.933122 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665e141-ef96-4092-a344-737ae6b0e425 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:06.933409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665e141-ef96-4092-a344-737ae6b0e425 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:06.933689 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665e141-ef96-4092-a344-737ae6b0e425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:06.949532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-6665e141-ef96-4092-a344-737ae6b0e425 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:06.950785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6665e141-ef96-4092-a344-737ae6b0e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:06.985784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:07.017103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 250/501] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:07.017961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665e141-ef96-4092-a344-737ae6b0e425 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.027076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae455d-b7b3-4508-af6a-565cff3343dc None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.062544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16a357-08a0-43e0-a171-a7afd272a387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.063343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16a357-08a0-43e0-a171-a7afd272a387 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.063629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16a357-08a0-43e0-a171-a7afd272a387 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.063905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16a357-08a0-43e0-a171-a7afd272a387 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.071946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16a357-08a0-43e0-a171-a7afd272a387 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:07.073403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16a357-08a0-43e0-a171-a7afd272a387 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-0d16a357-08a0-43e0-a171-a7afd272a387 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:07.087587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:07.088672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:07.089794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 252/502] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:07.090972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba74d06-d5ca-49ec-85f9-51e3358ffa38 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.091039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba74d06-d5ca-49ec-85f9-51e3358ffa38 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.091300 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba74d06-d5ca-49ec-85f9-51e3358ffa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.108808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-aba74d06-d5ca-49ec-85f9-51e3358ffa38 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:07.110055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-aba74d06-d5ca-49ec-85f9-51e3358ffa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:07.116624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:07.131068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665e141-ef96-4092-a344-737ae6b0e425 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.164961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83206b-e0b8-4afb-936c-1436e05b9a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.165753 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83206b-e0b8-4afb-936c-1436e05b9a9c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.166075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83206b-e0b8-4afb-936c-1436e05b9a9c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.166348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83206b-e0b8-4afb-936c-1436e05b9a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.176217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 253/503] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:07.176867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba74d06-d5ca-49ec-85f9-51e3358ffa38 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.178595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83206b-e0b8-4afb-936c-1436e05b9a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:07.180424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83206b-e0b8-4afb-936c-1436e05b9a9c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-2a83206b-e0b8-4afb-936c-1436e05b9a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:07.199217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:07.200292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:07.201505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:07.201505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:07.201505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:07.201505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:07.201505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 251/504] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:07.201854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83206b-e0b8-4afb-936c-1436e05b9a9c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.231758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:07.280814 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba74d06-d5ca-49ec-85f9-51e3358ffa38 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.303321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83206b-e0b8-4afb-936c-1436e05b9a9c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.314380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc737a18-92aa-41da-acec-8bb6a0354c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.315034 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc737a18-92aa-41da-acec-8bb6a0354c32 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.315322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc737a18-92aa-41da-acec-8bb6a0354c32 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.315602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc737a18-92aa-41da-acec-8bb6a0354c32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.337145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f374615-c487-4914-ae33-69616bff9740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.337965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f374615-c487-4914-ae33-69616bff9740 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.338258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f374615-c487-4914-ae33-69616bff9740 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.338531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f374615-c487-4914-ae33-69616bff9740 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.345895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:07.349339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f374615-c487-4914-ae33-69616bff9740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-1f374615-c487-4914-ae33-69616bff9740 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:07.357004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:07.358545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 252/505] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:07.359351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f374615-c487-4914-ae33-69616bff9740 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.388401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc737a18-92aa-41da-acec-8bb6a0354c32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:07.389909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc737a18-92aa-41da-acec-8bb6a0354c32 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:07.437297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:07.457660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f374615-c487-4914-ae33-69616bff9740 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.488464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 254/506] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:07.489187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc737a18-92aa-41da-acec-8bb6a0354c32 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.495107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda38890-1b6d-4b19-8338-6e2b0c9b2b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.495781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38890-1b6d-4b19-8338-6e2b0c9b2b69 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.496069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38890-1b6d-4b19-8338-6e2b0c9b2b69 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.496344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38890-1b6d-4b19-8338-6e2b0c9b2b69 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.497223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda38890-1b6d-4b19-8338-6e2b0c9b2b69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:07.497529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda38890-1b6d-4b19-8338-6e2b0c9b2b69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:07.498998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda38890-1b6d-4b19-8338-6e2b0c9b2b69 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:07.516557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 253/507] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:07.517343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda38890-1b6d-4b19-8338-6e2b0c9b2b69 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.572885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:07.594364 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc737a18-92aa-41da-acec-8bb6a0354c32 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.616141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda38890-1b6d-4b19-8338-6e2b0c9b2b69 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.628880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68717b-5816-482f-b82d-cff8ef1295bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.629665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68717b-5816-482f-b82d-cff8ef1295bd None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.629978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68717b-5816-482f-b82d-cff8ef1295bd None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.630254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68717b-5816-482f-b82d-cff8ef1295bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.640769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68717b-5816-482f-b82d-cff8ef1295bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:07.642241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68717b-5816-482f-b82d-cff8ef1295bd None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:07.649895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5acb6-25e7-4458-9465-95d1b8c532e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.650562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5acb6-25e7-4458-9465-95d1b8c532e8 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.650847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5acb6-25e7-4458-9465-95d1b8c532e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.651127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5acb6-25e7-4458-9465-95d1b8c532e8 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.651867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5acb6-25e7-4458-9465-95d1b8c532e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:07.652167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5acb6-25e7-4458-9465-95d1b8c532e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:07.654115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5acb6-25e7-4458-9465-95d1b8c532e8 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-bf68717b-5816-482f-b82d-cff8ef1295bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:07.659890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:07.660967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:07.661995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:07.661995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:07.661995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:07.661995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:07.661995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 255/508] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:07.661995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68717b-5816-482f-b82d-cff8ef1295bd None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.664798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:07.669860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 254/509] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:07.669860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5acb6-25e7-4458-9465-95d1b8c532e8 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.765040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68717b-5816-482f-b82d-cff8ef1295bd None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.768074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5acb6-25e7-4458-9465-95d1b8c532e8 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.783518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:07.797971 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6e840-fa30-4152-b934-6afb0496f985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.798640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e840-fa30-4152-b934-6afb0496f985 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.798923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e840-fa30-4152-b934-6afb0496f985 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.799199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6e840-fa30-4152-b934-6afb0496f985 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.799949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6e840-fa30-4152-b934-6afb0496f985 None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:07.800254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6e840-fa30-4152-b934-6afb0496f985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:07.800789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ec3b7-d4b8-452c-a42b-7768654959a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.801576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ec3b7-d4b8-452c-a42b-7768654959a7 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.801678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f6e840-fa30-4152-b934-6afb0496f985 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:07.801884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ec3b7-d4b8-452c-a42b-7768654959a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.802165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ec3b7-d4b8-452c-a42b-7768654959a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.810175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 256/510] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:07.810974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6e840-fa30-4152-b934-6afb0496f985 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.813281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ec3b7-d4b8-452c-a42b-7768654959a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:07.814754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ec3b7-d4b8-452c-a42b-7768654959a7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-374ec3b7-d4b8-452c-a42b-7768654959a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:07.833164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:07.834109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:07.834963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:07.834963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:07.834963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:07.834963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:07.834963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 255/511] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:07.835356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ec3b7-d4b8-452c-a42b-7768654959a7 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:07.877905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:07.915738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f6e840-fa30-4152-b934-6afb0496f985 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.933900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ec3b7-d4b8-452c-a42b-7768654959a7 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:07.950931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbf697-5884-42b0-aa0e-b14966547726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.951677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbf697-5884-42b0-aa0e-b14966547726 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.952031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbf697-5884-42b0-aa0e-b14966547726 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.952378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbf697-5884-42b0-aa0e-b14966547726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.967669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a750e-d8d1-4c41-8c94-e1e6ec2085bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:07.968324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a750e-d8d1-4c41-8c94-e1e6ec2085bc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:07.968605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a750e-d8d1-4c41-8c94-e1e6ec2085bc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:07.968886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a750e-d8d1-4c41-8c94-e1e6ec2085bc None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:07.969608 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a750e-d8d1-4c41-8c94-e1e6ec2085bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:07.969950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a750e-d8d1-4c41-8c94-e1e6ec2085bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:07.971308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a750e-d8d1-4c41-8c94-e1e6ec2085bc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:07.983073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 256/512] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:07.983969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a750e-d8d1-4c41-8c94-e1e6ec2085bc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.012355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:08.021595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbf697-5884-42b0-aa0e-b14966547726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.023087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbf697-5884-42b0-aa0e-b14966547726 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.083626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:08.087052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a750e-d8d1-4c41-8c94-e1e6ec2085bc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.120948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12e5ff-a6ce-4d60-9231-0c06bf35cfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.122324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12e5ff-a6ce-4d60-9231-0c06bf35cfea None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.122613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12e5ff-a6ce-4d60-9231-0c06bf35cfea None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.122706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 257/513] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:08.122892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12e5ff-a6ce-4d60-9231-0c06bf35cfea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:08.123481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbf697-5884-42b0-aa0e-b14966547726 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.130784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12e5ff-a6ce-4d60-9231-0c06bf35cfea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.132246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12e5ff-a6ce-4d60-9231-0c06bf35cfea None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-2f12e5ff-a6ce-4d60-9231-0c06bf35cfea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:08.148173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:08.149054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:08.149989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 257/514] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:08.151443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12e5ff-a6ce-4d60-9231-0c06bf35cfea None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.225171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbf697-5884-42b0-aa0e-b14966547726 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.239236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:08.251032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12e5ff-a6ce-4d60-9231-0c06bf35cfea None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.259698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa10c6-47bd-48bb-a40b-8013373bd5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.260487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa10c6-47bd-48bb-a40b-8013373bd5fe None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.260772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa10c6-47bd-48bb-a40b-8013373bd5fe None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.261048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa10c6-47bd-48bb-a40b-8013373bd5fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:08.271413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa10c6-47bd-48bb-a40b-8013373bd5fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.272858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa10c6-47bd-48bb-a40b-8013373bd5fe None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.285324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8157e-1e1f-49f0-8811-d74492dd0a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.286024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8157e-1e1f-49f0-8811-d74492dd0a56 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.286312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8157e-1e1f-49f0-8811-d74492dd0a56 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.286589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8157e-1e1f-49f0-8811-d74492dd0a56 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:08.287324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8157e-1e1f-49f0-8811-d74492dd0a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:08.287633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8157e-1e1f-49f0-8811-d74492dd0a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.288971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8157e-1e1f-49f0-8811-d74492dd0a56 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-69fa10c6-47bd-48bb-a40b-8013373bd5fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:08.290335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:08.291281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:08.292134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:08.292134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:08.292134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:08.292134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:08.292134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 258/515] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:08.292396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa10c6-47bd-48bb-a40b-8013373bd5fe None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.301023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 258/516] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:08.301854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8157e-1e1f-49f0-8811-d74492dd0a56 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.315655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:08.397276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa10c6-47bd-48bb-a40b-8013373bd5fe None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.401321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8157e-1e1f-49f0-8811-d74492dd0a56 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.431030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2b304-3f44-4a0d-876a-085a5526330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.431700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2b304-3f44-4a0d-876a-085a5526330a None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.431986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2b304-3f44-4a0d-876a-085a5526330a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.432261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2b304-3f44-4a0d-876a-085a5526330a None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:08.432963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2b304-3f44-4a0d-876a-085a5526330a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.434677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34741af-5e1c-4c6f-9891-a33c3268b9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.434934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2b304-3f44-4a0d-876a-085a5526330a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.435461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34741af-5e1c-4c6f-9891-a33c3268b9ff None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.435747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34741af-5e1c-4c6f-9891-a33c3268b9ff None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.436025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34741af-5e1c-4c6f-9891-a33c3268b9ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:08.446808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34741af-5e1c-4c6f-9891-a33c3268b9ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.448239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34741af-5e1c-4c6f-9891-a33c3268b9ff None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.456743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-d34741af-5e1c-4c6f-9891-a33c3268b9ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:08.466725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:08.467694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:08.468811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:08.468811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:08.468811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:08.468811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:08.468811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 259/517] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:08.468811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34741af-5e1c-4c6f-9891-a33c3268b9ff None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.537390 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 259/518] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:35:08 2022] POST /identity/v3/users => generated 247 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:08.538264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2b304-3f44-4a0d-876a-085a5526330a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.583081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34741af-5e1c-4c6f-9891-a33c3268b9ff None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.636954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65605718-a692-460a-90cf-e9e4f094ca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.637667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65605718-a692-460a-90cf-e9e4f094ca70 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.637991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65605718-a692-460a-90cf-e9e4f094ca70 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.638279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65605718-a692-460a-90cf-e9e4f094ca70 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:08.639194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65605718-a692-460a-90cf-e9e4f094ca70 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:08.639504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65605718-a692-460a-90cf-e9e4f094ca70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.640947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65605718-a692-460a-90cf-e9e4f094ca70 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.659789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:08.666061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 260/519] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 14 15:35:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:08.666061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65605718-a692-460a-90cf-e9e4f094ca70 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.699036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2b304-3f44-4a0d-876a-085a5526330a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.731894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:08.744733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4699744-c688-4d72-bcca-cb3f98e2254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.745496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4699744-c688-4d72-bcca-cb3f98e2254a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.745775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4699744-c688-4d72-bcca-cb3f98e2254a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.746103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4699744-c688-4d72-bcca-cb3f98e2254a None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:08.747369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4699744-c688-4d72-bcca-cb3f98e2254a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:08.747772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4699744-c688-4d72-bcca-cb3f98e2254a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.749190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4699744-c688-4d72-bcca-cb3f98e2254a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.763124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 260/520] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:08.764018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4699744-c688-4d72-bcca-cb3f98e2254a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.776704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65605718-a692-460a-90cf-e9e4f094ca70 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.812913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1d712-c998-4572-afa0-aa804e85e830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.813713 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1d712-c998-4572-afa0-aa804e85e830 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.814029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1d712-c998-4572-afa0-aa804e85e830 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.814307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1d712-c998-4572-afa0-aa804e85e830 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:08.821551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1d712-c998-4572-afa0-aa804e85e830 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.823034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1d712-c998-4572-afa0-aa804e85e830 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-9fa1d712-c998-4572-afa0-aa804e85e830 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:08.837889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:08.838777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:08.839622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:08.839622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:08.839622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:08.839622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:08.839622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:08.839622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:08.839622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:08.839622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:08.839622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 261/521] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:08.840255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1d712-c998-4572-afa0-aa804e85e830 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.880383 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4699744-c688-4d72-bcca-cb3f98e2254a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.912309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:08.917959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfa3e0-e152-4dc3-9f09-8bf091e02ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.924970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfa3e0-e152-4dc3-9f09-8bf091e02ebd None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.924970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfa3e0-e152-4dc3-9f09-8bf091e02ebd None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.924970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfa3e0-e152-4dc3-9f09-8bf091e02ebd None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:08.924970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfa3e0-e152-4dc3-9f09-8bf091e02ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:08.924970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfa3e0-e152-4dc3-9f09-8bf091e02ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:08.924970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfa3e0-e152-4dc3-9f09-8bf091e02ebd None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:08.938171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 261/522] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:08.938171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfa3e0-e152-4dc3-9f09-8bf091e02ebd None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:08.957141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1d712-c998-4572-afa0-aa804e85e830 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:08.977168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:08.996872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c38296-db7d-45eb-86ab-5b1ce39268cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:08.997599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c38296-db7d-45eb-86ab-5b1ce39268cf None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:08.997910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c38296-db7d-45eb-86ab-5b1ce39268cf None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:08.998191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c38296-db7d-45eb-86ab-5b1ce39268cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.050955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfa3e0-e152-4dc3-9f09-8bf091e02ebd None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:09.072058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c38296-db7d-45eb-86ab-5b1ce39268cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.073859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c38296-db7d-45eb-86ab-5b1ce39268cf None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.088509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd106c3-4c20-4a88-a8f7-ce61f33bce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:09.089310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd106c3-4c20-4a88-a8f7-ce61f33bce0c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:09.089601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd106c3-4c20-4a88-a8f7-ce61f33bce0c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:09.089892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd106c3-4c20-4a88-a8f7-ce61f33bce0c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.097960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd106c3-4c20-4a88-a8f7-ce61f33bce0c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.099406 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd106c3-4c20-4a88-a8f7-ce61f33bce0c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-9cd106c3-4c20-4a88-a8f7-ce61f33bce0c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:09.114473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:09.115411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 262/523] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:09.116400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd106c3-4c20-4a88-a8f7-ce61f33bce0c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:09.174994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:09.182944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 262/524] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:09.187669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c38296-db7d-45eb-86ab-5b1ce39268cf None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:09.203551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:09.230900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd106c3-4c20-4a88-a8f7-ce61f33bce0c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:09.269427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfdb07-43e8-4231-a1a1-a6bce4a0d616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:09.270239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfdb07-43e8-4231-a1a1-a6bce4a0d616 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:09.270523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfdb07-43e8-4231-a1a1-a6bce4a0d616 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:09.270805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfdb07-43e8-4231-a1a1-a6bce4a0d616 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.271600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfdb07-43e8-4231-a1a1-a6bce4a0d616 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:09.271912 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfdb07-43e8-4231-a1a1-a6bce4a0d616 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.273505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfdb07-43e8-4231-a1a1-a6bce4a0d616 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.291869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c38296-db7d-45eb-86ab-5b1ce39268cf None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:09.298150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 263/525] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:09.299312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfdb07-43e8-4231-a1a1-a6bce4a0d616 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:09.328719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb14d78-d746-428a-b6ff-6a3bab64f0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:09.329970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb14d78-d746-428a-b6ff-6a3bab64f0bb None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:09.330483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb14d78-d746-428a-b6ff-6a3bab64f0bb None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:09.330967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb14d78-d746-428a-b6ff-6a3bab64f0bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.337991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb14d78-d746-428a-b6ff-6a3bab64f0bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.339736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb14d78-d746-428a-b6ff-6a3bab64f0bb None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-9cb14d78-d746-428a-b6ff-6a3bab64f0bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:09.357406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:09.358379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:09.359245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:09.359245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:09.359245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:09.359245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:09.359245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:09.359245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:09.359245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:09.359245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:09.359245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 263/526] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:09.359717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb14d78-d746-428a-b6ff-6a3bab64f0bb None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:09.412499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfdb07-43e8-4231-a1a1-a6bce4a0d616 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:09.425739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:09.441570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:09.447458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db5642-e33d-4a7b-a7b7-ffbf27341fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:09.448141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db5642-e33d-4a7b-a7b7-ffbf27341fde None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:09.448423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db5642-e33d-4a7b-a7b7-ffbf27341fde None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:09.448838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db5642-e33d-4a7b-a7b7-ffbf27341fde None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.449560 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db5642-e33d-4a7b-a7b7-ffbf27341fde None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:09.449964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db5642-e33d-4a7b-a7b7-ffbf27341fde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.451721 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db5642-e33d-4a7b-a7b7-ffbf27341fde None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.463117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb14d78-d746-428a-b6ff-6a3bab64f0bb None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:09.490759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 264/527] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Jul 14 15:35:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:09.491961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db5642-e33d-4a7b-a7b7-ffbf27341fde None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:09.503575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5c82-b0f0-4b23-97d9-c9731be65685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:09.504376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5c82-b0f0-4b23-97d9-c9731be65685 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:09.504659 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5c82-b0f0-4b23-97d9-c9731be65685 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:09.504935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5c82-b0f0-4b23-97d9-c9731be65685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.516213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d5c82-b0f0-4b23-97d9-c9731be65685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.517660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d5c82-b0f0-4b23-97d9-c9731be65685 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-6e5d5c82-b0f0-4b23-97d9-c9731be65685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:09.535430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:09.536363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:09.537239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:09.537239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:09.537239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:09.537239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:09.537239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 264/528] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:09.537940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5c82-b0f0-4b23-97d9-c9731be65685 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:09.603757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db5642-e33d-4a7b-a7b7-ffbf27341fde None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:09.639350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fe207-4e32-4494-b747-cc914166d076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:09.640013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fe207-4e32-4494-b747-cc914166d076 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:09.640115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5c82-b0f0-4b23-97d9-c9731be65685 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:09.640299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fe207-4e32-4494-b747-cc914166d076 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:09.640569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fe207-4e32-4494-b747-cc914166d076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.641409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fe207-4e32-4494-b747-cc914166d076 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=ce88c973feef4627a1b871ee9cb26511) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:09.641705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fe207-4e32-4494-b747-cc914166d076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.643193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fe207-4e32-4494-b747-cc914166d076 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.649462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 265/529] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:09 2022] GET /identity/v3/role_assignments?user.id=ce88c973feef4627a1b871ee9cb26511&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:09.650292 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fe207-4e32-4494-b747-cc914166d076 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:09.688033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:09.696910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332ec5c-a381-4057-8884-96c8dfe8d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:09.697767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332ec5c-a381-4057-8884-96c8dfe8d596 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:09.701100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332ec5c-a381-4057-8884-96c8dfe8d596 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:09.701100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332ec5c-a381-4057-8884-96c8dfe8d596 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.701303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:09.702124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332ec5c-a381-4057-8884-96c8dfe8d596 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:09.702479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332ec5c-a381-4057-8884-96c8dfe8d596 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.703996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332ec5c-a381-4057-8884-96c8dfe8d596 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.732381 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 265/530] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:09.733597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332ec5c-a381-4057-8884-96c8dfe8d596 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:09.791320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fe207-4e32-4494-b747-cc914166d076 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:09.832619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97434-7f2f-49ac-8560-ed87f846507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:09.833622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97434-7f2f-49ac-8560-ed87f846507a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:09.834222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97434-7f2f-49ac-8560-ed87f846507a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:09.834698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d97434-7f2f-49ac-8560-ed87f846507a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.867556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332ec5c-a381-4057-8884-96c8dfe8d596 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:09.916629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97434-7f2f-49ac-8560-ed87f846507a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.916815 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96fa0a-d4b7-4b93-8649-2db6e0dcc51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:09.917501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96fa0a-d4b7-4b93-8649-2db6e0dcc51c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:09.917784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96fa0a-d4b7-4b93-8649-2db6e0dcc51c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:09.918089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96fa0a-d4b7-4b93-8649-2db6e0dcc51c None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:09.918865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96fa0a-d4b7-4b93-8649-2db6e0dcc51c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:09.919173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96fa0a-d4b7-4b93-8649-2db6e0dcc51c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:09.919752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d97434-7f2f-49ac-8560-ed87f846507a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.920542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96fa0a-d4b7-4b93-8649-2db6e0dcc51c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:09.940257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 266/531] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:09.940257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96fa0a-d4b7-4b93-8649-2db6e0dcc51c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:09.961732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:09.998948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:10.057292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96fa0a-d4b7-4b93-8649-2db6e0dcc51c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:10.066349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 266/532] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:10.067540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97434-7f2f-49ac-8560-ed87f846507a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:10.101108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b1718-163c-4031-9d2d-6558041d7ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:10.101782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b1718-163c-4031-9d2d-6558041d7ee4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:10.102090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b1718-163c-4031-9d2d-6558041d7ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:10.102379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b1718-163c-4031-9d2d-6558041d7ee4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:10.103485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b1718-163c-4031-9d2d-6558041d7ee4 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=5eeda3264191409eba35ffbb908770a6) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:10.103799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b1718-163c-4031-9d2d-6558041d7ee4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:10.106080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b1718-163c-4031-9d2d-6558041d7ee4 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:10.117097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 267/533] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:09 2022] GET /identity/v3/role_assignments?user.id=5eeda3264191409eba35ffbb908770a6&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:10.118302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b1718-163c-4031-9d2d-6558041d7ee4 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:10.232430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d97434-7f2f-49ac-8560-ed87f846507a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:10.287612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef37d4-7ceb-471f-80d6-fba4ec01d73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:10.288450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef37d4-7ceb-471f-80d6-fba4ec01d73f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:10.288736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef37d4-7ceb-471f-80d6-fba4ec01d73f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:10.289009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef37d4-7ceb-471f-80d6-fba4ec01d73f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:10.291330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:10.301850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef37d4-7ceb-471f-80d6-fba4ec01d73f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:10.307729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef37d4-7ceb-471f-80d6-fba4ec01d73f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:10.333209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b1718-163c-4031-9d2d-6558041d7ee4 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-c6ef37d4-7ceb-471f-80d6-fba4ec01d73f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:10.339055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:10.339926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:10.341341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:10.341341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:10.341341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:10.341341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:10.341341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:10.341341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:10.341341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:10.341341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:10.344419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 267/534] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:10.350404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef37d4-7ceb-471f-80d6-fba4ec01d73f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:10.367006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:10.391896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12131288-5105-48ab-9b35-09177c52c61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:10.393061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12131288-5105-48ab-9b35-09177c52c61c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:10.393668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12131288-5105-48ab-9b35-09177c52c61c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:10.394185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12131288-5105-48ab-9b35-09177c52c61c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:10.406386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12131288-5105-48ab-9b35-09177c52c61c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:10.409127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12131288-5105-48ab-9b35-09177c52c61c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-12131288-5105-48ab-9b35-09177c52c61c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:10.427596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:10.428541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:10.429497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:10.429497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:10.429497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:10.429497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:10.431746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 268/535] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:10.437290 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12131288-5105-48ab-9b35-09177c52c61c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:10.520190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef37d4-7ceb-471f-80d6-fba4ec01d73f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:10.552350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12131288-5105-48ab-9b35-09177c52c61c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:10.554853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52a05a-ef54-4cd8-bbaf-2001d3842e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:10.555541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a05a-ef54-4cd8-bbaf-2001d3842e4e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:10.555848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a05a-ef54-4cd8-bbaf-2001d3842e4e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:10.556882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52a05a-ef54-4cd8-bbaf-2001d3842e4e None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:10.556882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a05a-ef54-4cd8-bbaf-2001d3842e4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:10.557204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a05a-ef54-4cd8-bbaf-2001d3842e4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:10.558699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52a05a-ef54-4cd8-bbaf-2001d3842e4e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:10.572862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 268/536] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:10.586120 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52a05a-ef54-4cd8-bbaf-2001d3842e4e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:10.597552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9dad2a-9a12-4452-9893-a59b55880d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:10.598547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9dad2a-9a12-4452-9893-a59b55880d7b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:10.599074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9dad2a-9a12-4452-9893-a59b55880d7b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:10.599587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9dad2a-9a12-4452-9893-a59b55880d7b None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:10.600560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9dad2a-9a12-4452-9893-a59b55880d7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:10.601084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9dad2a-9a12-4452-9893-a59b55880d7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:10.602685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9dad2a-9a12-4452-9893-a59b55880d7b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:10.616429 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 269/537] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:10.622708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9dad2a-9a12-4452-9893-a59b55880d7b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:10.687886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:10.694213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:10.738256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9dad2a-9a12-4452-9893-a59b55880d7b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:10.738745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52a05a-ef54-4cd8-bbaf-2001d3842e4e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:10.778620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d57f5-9f03-463b-83f0-e6898d7d3b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:10.779352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d57f5-9f03-463b-83f0-e6898d7d3b5d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:10.779635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d57f5-9f03-463b-83f0-e6898d7d3b5d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:10.779915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d57f5-9f03-463b-83f0-e6898d7d3b5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:10.781284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93866542-7c27-439c-b699-52a569f9041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:10.782391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93866542-7c27-439c-b699-52a569f9041f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:10.783248 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93866542-7c27-439c-b699-52a569f9041f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:10.783678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93866542-7c27-439c-b699-52a569f9041f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:10.784660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93866542-7c27-439c-b699-52a569f9041f None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=3d54d684efce44b99f3235e80beb1c56) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:10.785124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93866542-7c27-439c-b699-52a569f9041f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:10.786984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d57f5-9f03-463b-83f0-e6898d7d3b5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:10.787217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93866542-7c27-439c-b699-52a569f9041f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:10.788430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d57f5-9f03-463b-83f0-e6898d7d3b5d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:10.794520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 269/538] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:10 2022] GET /identity/v3/role_assignments?user.id=3d54d684efce44b99f3235e80beb1c56&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:10.796377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652dded4-13ae-43ec-9ba6-cab069726c6a None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:10.796818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652dded4-13ae-43ec-9ba6-cab069726c6a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:10.797267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652dded4-13ae-43ec-9ba6-cab069726c6a None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:10.803140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 270/539] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-8a7d57f5-9f03-463b-83f0-e6898d7d3b5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:10.803565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:10.804555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:10.805542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:10.805542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:10.805542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:10.805542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:10.805542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:10.805542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:10.805542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:10.805542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:10.805542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 270/540] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:10.807561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d786fb-5ac4-429f-9ac2-e4f7afcd8a2f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:10.807853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d786fb-5ac4-429f-9ac2-e4f7afcd8a2f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:10.808138 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d786fb-5ac4-429f-9ac2-e4f7afcd8a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:10.819002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d57f5-9f03-463b-83f0-e6898d7d3b5d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:10.843445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d1d786fb-5ac4-429f-9ac2-e4f7afcd8a2f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:10.844694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d1d786fb-5ac4-429f-9ac2-e4f7afcd8a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:10.969218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 271/541] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:10.993243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:10.994490 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d786fb-5ac4-429f-9ac2-e4f7afcd8a2f None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:11.010679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d57f5-9f03-463b-83f0-e6898d7d3b5d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:11.054139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20879ba2-013a-4b3a-ad37-bf105f7678fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:11.054139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20879ba2-013a-4b3a-ad37-bf105f7678fc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:11.054139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20879ba2-013a-4b3a-ad37-bf105f7678fc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:11.054139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20879ba2-013a-4b3a-ad37-bf105f7678fc None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:11.054545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20879ba2-013a-4b3a-ad37-bf105f7678fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:11.054545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20879ba2-013a-4b3a-ad37-bf105f7678fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:11.054545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20879ba2-013a-4b3a-ad37-bf105f7678fc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:11.068333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 271/542] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:11.074949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20879ba2-013a-4b3a-ad37-bf105f7678fc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:11.094184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:11.182185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d786fb-5ac4-429f-9ac2-e4f7afcd8a2f None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:11.239848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53047275-9af7-4f72-856c-c42adb6ceb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:11.240653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53047275-9af7-4f72-856c-c42adb6ceb9e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:11.240938 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53047275-9af7-4f72-856c-c42adb6ceb9e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:11.241213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53047275-9af7-4f72-856c-c42adb6ceb9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:11.241483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20879ba2-013a-4b3a-ad37-bf105f7678fc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:11.263643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53047275-9af7-4f72-856c-c42adb6ceb9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:11.279161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52c5f9-8072-4ae1-adbe-50f0ab3b0f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-53047275-9af7-4f72-856c-c42adb6ceb9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:11.279613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:11.280478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:11.281306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 272/543] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:11.283064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52c5f9-8072-4ae1-adbe-50f0ab3b0f08 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:11.283742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52c5f9-8072-4ae1-adbe-50f0ab3b0f08 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:11.284399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52c5f9-8072-4ae1-adbe-50f0ab3b0f08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:11.286033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52c5f9-8072-4ae1-adbe-50f0ab3b0f08 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=7e63b8e907764f2c818391ab2ad9184c) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:11.286513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52c5f9-8072-4ae1-adbe-50f0ab3b0f08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:11.288091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52c5f9-8072-4ae1-adbe-50f0ab3b0f08 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:11.290724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53047275-9af7-4f72-856c-c42adb6ceb9e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:11.295544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 272/544] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:11 2022] GET /identity/v3/role_assignments?user.id=7e63b8e907764f2c818391ab2ad9184c&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:11.404252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53047275-9af7-4f72-856c-c42adb6ceb9e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:11.414348 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:11.453542 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55794e76-caec-4d59-98fc-c4bc69ba86b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:11.454272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794e76-caec-4d59-98fc-c4bc69ba86b0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:11.454547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794e76-caec-4d59-98fc-c4bc69ba86b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:11.454830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794e76-caec-4d59-98fc-c4bc69ba86b0 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:11.455594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794e76-caec-4d59-98fc-c4bc69ba86b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:11.455910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794e76-caec-4d59-98fc-c4bc69ba86b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:11.457333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794e76-caec-4d59-98fc-c4bc69ba86b0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:11.473391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52c5f9-8072-4ae1-adbe-50f0ab3b0f08 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:11.474187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 273/545] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:11.500819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:11.610104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52c5f9-8072-4ae1-adbe-50f0ab3b0f08 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:11.648334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b516b7-65ee-403c-aaed-2963e139f2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:11.649507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b516b7-65ee-403c-aaed-2963e139f2c9 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:11.650032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b516b7-65ee-403c-aaed-2963e139f2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:11.650515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b516b7-65ee-403c-aaed-2963e139f2c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:11.662704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b516b7-65ee-403c-aaed-2963e139f2c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:11.664492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b516b7-65ee-403c-aaed-2963e139f2c9 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-a1b516b7-65ee-403c-aaed-2963e139f2c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:11.684206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:11.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:11.686076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:11.686076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:11.686076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:11.686076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:11.688288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 273/546] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:11.694922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55794e76-caec-4d59-98fc-c4bc69ba86b0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:11.834253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55794e76-caec-4d59-98fc-c4bc69ba86b0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:11.881338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc3c0f-1a54-4c97-a776-7817f9238e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:11.881962 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc3c0f-1a54-4c97-a776-7817f9238e8d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:11.882253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc3c0f-1a54-4c97-a776-7817f9238e8d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:11.882535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc3c0f-1a54-4c97-a776-7817f9238e8d None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:11.883266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc3c0f-1a54-4c97-a776-7817f9238e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:11.883576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc3c0f-1a54-4c97-a776-7817f9238e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:11.884972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc3c0f-1a54-4c97-a776-7817f9238e8d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:11.901899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 274/547] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:11.907553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b516b7-65ee-403c-aaed-2963e139f2c9 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:11.989689 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:12.029446 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b516b7-65ee-403c-aaed-2963e139f2c9 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:12.067020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda76c0d-601a-40ad-ae71-838f6cff7694 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:12.067307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda76c0d-601a-40ad-ae71-838f6cff7694 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:12.067588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda76c0d-601a-40ad-ae71-838f6cff7694 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:12.068959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 275/548] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:12.074452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a75d56-805f-4cd9-bd3a-c72893e48ec8 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:12.075608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a75d56-805f-4cd9-bd3a-c72893e48ec8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:12.076088 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a75d56-805f-4cd9-bd3a-c72893e48ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:12.087154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70933da1-17c0-4111-8826-6e936c6cbc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:12.087961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70933da1-17c0-4111-8826-6e936c6cbc18 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:12.088245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70933da1-17c0-4111-8826-6e936c6cbc18 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:12.088522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70933da1-17c0-4111-8826-6e936c6cbc18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:12.096840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70933da1-17c0-4111-8826-6e936c6cbc18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:12.098334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70933da1-17c0-4111-8826-6e936c6cbc18 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:12.119090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-65a75d56-805f-4cd9-bd3a-c72893e48ec8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:12.120329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-65a75d56-805f-4cd9-bd3a-c72893e48ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-70933da1-17c0-4111-8826-6e936c6cbc18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:12.128312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:12.129410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:12.130354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:12.132373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 274/549] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:12.140294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70933da1-17c0-4111-8826-6e936c6cbc18 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:12.172828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:12.254435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 276/550] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:35:12.274788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a75d56-805f-4cd9-bd3a-c72893e48ec8 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:12.345269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70933da1-17c0-4111-8826-6e936c6cbc18 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:12.376077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:12.395461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e644be3-dec9-43e2-ade2-c8e9b97ce743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:12.395461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e644be3-dec9-43e2-ade2-c8e9b97ce743 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:12.395461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e644be3-dec9-43e2-ade2-c8e9b97ce743 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:12.395461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e644be3-dec9-43e2-ade2-c8e9b97ce743 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:12.395816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e644be3-dec9-43e2-ade2-c8e9b97ce743 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:12.395816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e644be3-dec9-43e2-ade2-c8e9b97ce743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:12.396188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e644be3-dec9-43e2-ade2-c8e9b97ce743 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:12.433203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 275/551] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 14 15:35:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:12.438250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e644be3-dec9-43e2-ade2-c8e9b97ce743 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:12.476608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a75d56-805f-4cd9-bd3a-c72893e48ec8 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:12.566525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7814ce-793c-45f7-8668-39867a83e990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:12.567333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7814ce-793c-45f7-8668-39867a83e990 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:12.567617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7814ce-793c-45f7-8668-39867a83e990 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:12.567887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7814ce-793c-45f7-8668-39867a83e990 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:12.578812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e644be3-dec9-43e2-ade2-c8e9b97ce743 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:12.596261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7814ce-793c-45f7-8668-39867a83e990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-0f7814ce-793c-45f7-8668-39867a83e990 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:12.604057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:12.604969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:12.605799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 277/552] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:12.625772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7814ce-793c-45f7-8668-39867a83e990 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:12.645004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f27b5a-c847-47eb-944d-1f98c21e19e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:12.646269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f27b5a-c847-47eb-944d-1f98c21e19e0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:12.646646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f27b5a-c847-47eb-944d-1f98c21e19e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:12.647159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f27b5a-c847-47eb-944d-1f98c21e19e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:12.653804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:12.714015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:12.740283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f27b5a-c847-47eb-944d-1f98c21e19e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:12.741770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f27b5a-c847-47eb-944d-1f98c21e19e0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:12.760483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7814ce-793c-45f7-8668-39867a83e990 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:12.809327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37572dea-a514-497a-9acb-79ff777a74cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:12.811059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37572dea-a514-497a-9acb-79ff777a74cf None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:12.811903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37572dea-a514-497a-9acb-79ff777a74cf None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:12.812611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37572dea-a514-497a-9acb-79ff777a74cf None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:12.813788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37572dea-a514-497a-9acb-79ff777a74cf None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:12.814668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37572dea-a514-497a-9acb-79ff777a74cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:12.816826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37572dea-a514-497a-9acb-79ff777a74cf None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:12.828835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 278/553] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:12.830423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2a26a-3db1-4719-a6fe-d29880e14b32 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:12.830867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2a26a-3db1-4719-a6fe-d29880e14b32 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:12.831319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2a26a-3db1-4719-a6fe-d29880e14b32 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:12.832990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 279/554] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:12.833871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2a26a-3db1-4719-a6fe-d29880e14b32 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:12.867334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 276/555] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:12.868666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e24cc1-7e35-40d7-a7bc-223bd6caad52 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:12.869033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e24cc1-7e35-40d7-a7bc-223bd6caad52 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:12.869229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e24cc1-7e35-40d7-a7bc-223bd6caad52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:12.885536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-80e24cc1-7e35-40d7-a7bc-223bd6caad52 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:12.886752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-80e24cc1-7e35-40d7-a7bc-223bd6caad52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:12.948414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:12.957194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 277/556] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:12.958146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e24cc1-7e35-40d7-a7bc-223bd6caad52 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:12.971329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:12.982101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2a26a-3db1-4719-a6fe-d29880e14b32 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.021426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac536e-5df2-4fe4-8fef-1a84d5aa031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.022255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac536e-5df2-4fe4-8fef-1a84d5aa031b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.022534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac536e-5df2-4fe4-8fef-1a84d5aa031b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.023008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac536e-5df2-4fe4-8fef-1a84d5aa031b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.035345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac536e-5df2-4fe4-8fef-1a84d5aa031b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:13.037721 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac536e-5df2-4fe4-8fef-1a84d5aa031b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-35ac536e-5df2-4fe4-8fef-1a84d5aa031b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:13.059812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:13.061267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:13.062188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:13.062188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:13.062188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:13.062188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:13.062188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 280/557] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:13.067539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac536e-5df2-4fe4-8fef-1a84d5aa031b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.092625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e24cc1-7e35-40d7-a7bc-223bd6caad52 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.130905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb263ffc-0979-4a78-9106-28ce57f59fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.131860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb263ffc-0979-4a78-9106-28ce57f59fab None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.132130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb263ffc-0979-4a78-9106-28ce57f59fab None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.132496 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb263ffc-0979-4a78-9106-28ce57f59fab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.144275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb263ffc-0979-4a78-9106-28ce57f59fab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:13.146102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb263ffc-0979-4a78-9106-28ce57f59fab None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-fb263ffc-0979-4a78-9106-28ce57f59fab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:13.164389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:13.165785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:13.166698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:13.166698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:13.166698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:13.166698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:13.170799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 278/558] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:13.171350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb263ffc-0979-4a78-9106-28ce57f59fab None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.201076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac536e-5df2-4fe4-8fef-1a84d5aa031b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.234169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee501ea1-5f35-4095-93cf-ccbdc4ee981e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.234973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501ea1-5f35-4095-93cf-ccbdc4ee981e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.235263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501ea1-5f35-4095-93cf-ccbdc4ee981e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.235546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee501ea1-5f35-4095-93cf-ccbdc4ee981e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.237440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:13.238390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:13.247438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee501ea1-5f35-4095-93cf-ccbdc4ee981e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-ee501ea1-5f35-4095-93cf-ccbdc4ee981e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:13.255196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:13.256675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 281/559] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:13.257636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee501ea1-5f35-4095-93cf-ccbdc4ee981e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.282804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb263ffc-0979-4a78-9106-28ce57f59fab None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.319667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f73ea-50d8-4a6a-8a0f-9ad0a6048b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.320358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f73ea-50d8-4a6a-8a0f-9ad0a6048b3c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.320636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f73ea-50d8-4a6a-8a0f-9ad0a6048b3c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.320918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f73ea-50d8-4a6a-8a0f-9ad0a6048b3c None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.321654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f73ea-50d8-4a6a-8a0f-9ad0a6048b3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:13.322008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f73ea-50d8-4a6a-8a0f-9ad0a6048b3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:13.323386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f73ea-50d8-4a6a-8a0f-9ad0a6048b3c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:13.335671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 279/560] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:13.336883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b5fad-daee-4492-8e23-03d8dca5475f None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.337178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b5fad-daee-4492-8e23-03d8dca5475f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.337458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b5fad-daee-4492-8e23-03d8dca5475f None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.342956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 280/561] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:13.343789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b5fad-daee-4492-8e23-03d8dca5475f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.383480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee501ea1-5f35-4095-93cf-ccbdc4ee981e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.423904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fde318-b074-45f3-ab0a-adb13f3e1006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.424576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fde318-b074-45f3-ab0a-adb13f3e1006 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.424910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fde318-b074-45f3-ab0a-adb13f3e1006 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.425308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fde318-b074-45f3-ab0a-adb13f3e1006 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.426061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fde318-b074-45f3-ab0a-adb13f3e1006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:13.426374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fde318-b074-45f3-ab0a-adb13f3e1006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:13.427893 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fde318-b074-45f3-ab0a-adb13f3e1006 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:13.446106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 282/562] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:13.446106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fde318-b074-45f3-ab0a-adb13f3e1006 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.455233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b5fad-daee-4492-8e23-03d8dca5475f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.477602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:13.490582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e47db8-26ce-4692-a85a-bdafba871f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.491327 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e47db8-26ce-4692-a85a-bdafba871f61 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.491614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e47db8-26ce-4692-a85a-bdafba871f61 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.491894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e47db8-26ce-4692-a85a-bdafba871f61 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.492703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e47db8-26ce-4692-a85a-bdafba871f61 None admin] RBAC: Adding query filter params (name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:13.493016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e47db8-26ce-4692-a85a-bdafba871f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:13.494431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e47db8-26ce-4692-a85a-bdafba871f61 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:13.507571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 281/563] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:13.507571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d0b4f-bd32-4185-b876-592c31770fbf None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.507571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d0b4f-bd32-4185-b876-592c31770fbf None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.507571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d0b4f-bd32-4185-b876-592c31770fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.509054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:13.521727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ae7d0b4f-bd32-4185-b876-592c31770fbf None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:13.522940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ae7d0b4f-bd32-4185-b876-592c31770fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:13.557524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fde318-b074-45f3-ab0a-adb13f3e1006 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.594466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cb32b-5cde-4149-b81d-5b3fbd3481d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.595376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cb32b-5cde-4149-b81d-5b3fbd3481d3 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.595666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cb32b-5cde-4149-b81d-5b3fbd3481d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.595903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 282/564] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:13.606393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cb32b-5cde-4149-b81d-5b3fbd3481d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.606477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d0b4f-bd32-4185-b876-592c31770fbf None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.606576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cb32b-5cde-4149-b81d-5b3fbd3481d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:13.607770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cb32b-5cde-4149-b81d-5b3fbd3481d3 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-4f9cb32b-5cde-4149-b81d-5b3fbd3481d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:13.625434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:13.626827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 283/565] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:13.628210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cb32b-5cde-4149-b81d-5b3fbd3481d3 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.715619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d0b4f-bd32-4185-b876-592c31770fbf None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.728685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:13.728807 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cb32b-5cde-4149-b81d-5b3fbd3481d3 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.742689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:13.748903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb3599-ab47-44fd-956c-1d13342e21e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.749706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb3599-ab47-44fd-956c-1d13342e21e6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.750019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb3599-ab47-44fd-956c-1d13342e21e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.750297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb3599-ab47-44fd-956c-1d13342e21e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.756908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb3599-ab47-44fd-956c-1d13342e21e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:13.758464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb3599-ab47-44fd-956c-1d13342e21e6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:13.763289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8816452-6d66-4f3f-86b6-cf2e54efb5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.764081 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8816452-6d66-4f3f-86b6-cf2e54efb5d8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.764367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8816452-6d66-4f3f-86b6-cf2e54efb5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.764645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8816452-6d66-4f3f-86b6-cf2e54efb5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-b9eb3599-ab47-44fd-956c-1d13342e21e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:13.773932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:13.774837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:13.775746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:13.775746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:13.775746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:13.775746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:13.775746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:13.775746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:13.775746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:13.775746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:13.775746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 283/566] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:13.776233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8816452-6d66-4f3f-86b6-cf2e54efb5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:13.776304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb3599-ab47-44fd-956c-1d13342e21e6 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.776886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8816452-6d66-4f3f-86b6-cf2e54efb5d8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-a8816452-6d66-4f3f-86b6-cf2e54efb5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:13.794691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:13.795786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:13.796703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:13.796703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:13.796703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:13.796703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:13.798695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 284/567] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:13.799489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8816452-6d66-4f3f-86b6-cf2e54efb5d8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.876167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb3599-ab47-44fd-956c-1d13342e21e6 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.898662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8816452-6d66-4f3f-86b6-cf2e54efb5d8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:13.909655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64611aad-d846-4b30-9ba7-1b77e13efab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.910477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64611aad-d846-4b30-9ba7-1b77e13efab1 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.910766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64611aad-d846-4b30-9ba7-1b77e13efab1 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.911045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64611aad-d846-4b30-9ba7-1b77e13efab1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.921079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64611aad-d846-4b30-9ba7-1b77e13efab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-64611aad-d846-4b30-9ba7-1b77e13efab1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:13.928754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:13.929872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:13.930912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 284/568] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:13.935710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bab85-342d-4e42-8cad-6935612fd13d None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.935906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc6f1c-c60f-480b-98c3-14d289cec34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:13.936025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bab85-342d-4e42-8cad-6935612fd13d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.936267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bab85-342d-4e42-8cad-6935612fd13d None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.936574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc6f1c-c60f-480b-98c3-14d289cec34e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:13.936857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc6f1c-c60f-480b-98c3-14d289cec34e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:13.937135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc6f1c-c60f-480b-98c3-14d289cec34e None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:13.937566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 285/569] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:13.938005 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc6f1c-c60f-480b-98c3-14d289cec34e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:13.938306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bab85-342d-4e42-8cad-6935612fd13d None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.938397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc6f1c-c60f-480b-98c3-14d289cec34e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:13.939754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc6f1c-c60f-480b-98c3-14d289cec34e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:13.958499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 285/570] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:13.959311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc6f1c-c60f-480b-98c3-14d289cec34e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:13.974016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:13.984195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:14.045399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bab85-342d-4e42-8cad-6935612fd13d None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.058919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc6f1c-c60f-480b-98c3-14d289cec34e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.116184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8f055-f211-415f-9929-d759c29d083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.116844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c61fdc-4238-47fc-8283-4eea4b45c3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.116969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f055-f211-415f-9929-d759c29d083a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.117131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f055-f211-415f-9929-d759c29d083a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.117411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8f055-f211-415f-9929-d759c29d083a None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.117504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c61fdc-4238-47fc-8283-4eea4b45c3a1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.117766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c61fdc-4238-47fc-8283-4eea4b45c3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.118065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c61fdc-4238-47fc-8283-4eea4b45c3a1 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.118236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8f055-f211-415f-9929-d759c29d083a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:14.118550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8f055-f211-415f-9929-d759c29d083a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:14.118783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c61fdc-4238-47fc-8283-4eea4b45c3a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:14.119085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c61fdc-4238-47fc-8283-4eea4b45c3a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:14.119977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8f055-f211-415f-9929-d759c29d083a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:14.120441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c61fdc-4238-47fc-8283-4eea4b45c3a1 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:14.131607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 286/571] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:14.136663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 286/572] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:14.136827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c61fdc-4238-47fc-8283-4eea4b45c3a1 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.136933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c497e3-0019-48c6-a7dc-2e41ef561d7d None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.136933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c497e3-0019-48c6-a7dc-2e41ef561d7d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.136933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c497e3-0019-48c6-a7dc-2e41ef561d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.151908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d0c497e3-0019-48c6-a7dc-2e41ef561d7d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:14.153680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d0c497e3-0019-48c6-a7dc-2e41ef561d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:14.219129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:14.219629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 287/573] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:14.220451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c497e3-0019-48c6-a7dc-2e41ef561d7d None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.238675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c61fdc-4238-47fc-8283-4eea4b45c3a1 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.245368 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:14.273257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88443617-5240-4393-bca6-4c7730f36ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.273963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88443617-5240-4393-bca6-4c7730f36ea5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.274252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88443617-5240-4393-bca6-4c7730f36ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.274530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88443617-5240-4393-bca6-4c7730f36ea5 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.275272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88443617-5240-4393-bca6-4c7730f36ea5 None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:14.275584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88443617-5240-4393-bca6-4c7730f36ea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:14.276957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88443617-5240-4393-bca6-4c7730f36ea5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:14.285454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 287/574] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:14.286319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88443617-5240-4393-bca6-4c7730f36ea5 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.319731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c497e3-0019-48c6-a7dc-2e41ef561d7d None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.353000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a2f64-e0d1-4a83-8b00-259646674d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.353793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a2f64-e0d1-4a83-8b00-259646674d67 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.354110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a2f64-e0d1-4a83-8b00-259646674d67 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.354387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a2f64-e0d1-4a83-8b00-259646674d67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.364886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a2f64-e0d1-4a83-8b00-259646674d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:14.366367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a2f64-e0d1-4a83-8b00-259646674d67 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-0d0a2f64-e0d1-4a83-8b00-259646674d67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:14.384062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:14.385146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:14.390709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:14.390709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:14.390709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:14.390709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:14.390709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 288/575] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:14.390709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a2f64-e0d1-4a83-8b00-259646674d67 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.391880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88443617-5240-4393-bca6-4c7730f36ea5 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.425596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a977f2-bb82-404b-8287-82fa421c1b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.426409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a977f2-bb82-404b-8287-82fa421c1b61 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.426699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a977f2-bb82-404b-8287-82fa421c1b61 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.426978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a977f2-bb82-404b-8287-82fa421c1b61 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.434399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a977f2-bb82-404b-8287-82fa421c1b61 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:14.435844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a977f2-bb82-404b-8287-82fa421c1b61 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-41a977f2-bb82-404b-8287-82fa421c1b61 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:14.449997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:14.451119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 288/576] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:14.452243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a977f2-bb82-404b-8287-82fa421c1b61 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.464103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:14.466493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:14.488521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a2f64-e0d1-4a83-8b00-259646674d67 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.521452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeda13f-315d-4b80-bf45-0559cc0d3e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.522169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeda13f-315d-4b80-bf45-0559cc0d3e22 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.522456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeda13f-315d-4b80-bf45-0559cc0d3e22 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.522731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeda13f-315d-4b80-bf45-0559cc0d3e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.523558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeda13f-315d-4b80-bf45-0559cc0d3e22 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=95f36b4517ee4b64bc43e000c854df30) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:14.523868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeda13f-315d-4b80-bf45-0559cc0d3e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:14.525287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeda13f-315d-4b80-bf45-0559cc0d3e22 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:14.531946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 289/577] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/role_assignments?user.id=95f36b4517ee4b64bc43e000c854df30&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:14.532879 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeda13f-315d-4b80-bf45-0559cc0d3e22 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.563451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a977f2-bb82-404b-8287-82fa421c1b61 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.605899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a938bd-8062-42d0-ba8f-c8b372e31596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.606722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a938bd-8062-42d0-ba8f-c8b372e31596 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.607007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a938bd-8062-42d0-ba8f-c8b372e31596 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.607283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a938bd-8062-42d0-ba8f-c8b372e31596 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.619686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a938bd-8062-42d0-ba8f-c8b372e31596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-e3a938bd-8062-42d0-ba8f-c8b372e31596 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:14.628228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:14.629421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 289/578] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:14.630333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a938bd-8062-42d0-ba8f-c8b372e31596 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.659961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeda13f-315d-4b80-bf45-0559cc0d3e22 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.698417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24121a73-87be-4633-850f-7adba8f3249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.699374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24121a73-87be-4633-850f-7adba8f3249b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.700322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24121a73-87be-4633-850f-7adba8f3249b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.700698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24121a73-87be-4633-850f-7adba8f3249b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.710019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:14.712818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24121a73-87be-4633-850f-7adba8f3249b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:14.714316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24121a73-87be-4633-850f-7adba8f3249b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:14.715775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-24121a73-87be-4633-850f-7adba8f3249b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:14.731868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:14.732998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:14.738791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:14.738791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:14.738791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:14.738791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:14.738791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 290/579] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:14.738791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24121a73-87be-4633-850f-7adba8f3249b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.776184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a938bd-8062-42d0-ba8f-c8b372e31596 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.832474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feb3ee-e06b-4694-bb69-c93609a2d705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.833625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feb3ee-e06b-4694-bb69-c93609a2d705 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.834162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feb3ee-e06b-4694-bb69-c93609a2d705 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.834666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feb3ee-e06b-4694-bb69-c93609a2d705 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.836064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feb3ee-e06b-4694-bb69-c93609a2d705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:14.836624 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feb3ee-e06b-4694-bb69-c93609a2d705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:14.839336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feb3ee-e06b-4694-bb69-c93609a2d705 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:14.854028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24121a73-87be-4633-850f-7adba8f3249b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.859432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 290/580] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:14.860498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feb3ee-e06b-4694-bb69-c93609a2d705 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.906763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6fae4-1e71-401d-8de0-3309e71caf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:14.907440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6fae4-1e71-401d-8de0-3309e71caf1b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:14.907721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6fae4-1e71-401d-8de0-3309e71caf1b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:14.907996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6fae4-1e71-401d-8de0-3309e71caf1b None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:14.908834 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6fae4-1e71-401d-8de0-3309e71caf1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:14.909144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6fae4-1e71-401d-8de0-3309e71caf1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:14.910749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6fae4-1e71-401d-8de0-3309e71caf1b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:14.936430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 291/581] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:14.936430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6fae4-1e71-401d-8de0-3309e71caf1b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:14.977488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feb3ee-e06b-4694-bb69-c93609a2d705 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:14.986363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:15.011409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:15.020905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044fba5-9dc0-4346-a188-13022b24edca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.021587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044fba5-9dc0-4346-a188-13022b24edca None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.021901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044fba5-9dc0-4346-a188-13022b24edca None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.022182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044fba5-9dc0-4346-a188-13022b24edca None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.022941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044fba5-9dc0-4346-a188-13022b24edca None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:15.023251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044fba5-9dc0-4346-a188-13022b24edca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.024645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044fba5-9dc0-4346-a188-13022b24edca None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.039331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 291/582] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:15.040606 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044fba5-9dc0-4346-a188-13022b24edca None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.048625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6fae4-1e71-401d-8de0-3309e71caf1b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.085772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1db652-27f2-420b-ae7a-769e4a0ca2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.086575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1db652-27f2-420b-ae7a-769e4a0ca2cc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.086863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1db652-27f2-420b-ae7a-769e4a0ca2cc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.087138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1db652-27f2-420b-ae7a-769e4a0ca2cc None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.087887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1db652-27f2-420b-ae7a-769e4a0ca2cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:15.088171 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1db652-27f2-420b-ae7a-769e4a0ca2cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.089529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1db652-27f2-420b-ae7a-769e4a0ca2cc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.106852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 292/583] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:15.106852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1db652-27f2-420b-ae7a-769e4a0ca2cc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.150882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044fba5-9dc0-4346-a188-13022b24edca None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.186021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f07224-295e-4e76-a27d-af37d2bec623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.186817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f07224-295e-4e76-a27d-af37d2bec623 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.187100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f07224-295e-4e76-a27d-af37d2bec623 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.187375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f07224-295e-4e76-a27d-af37d2bec623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.194796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f07224-295e-4e76-a27d-af37d2bec623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.196244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f07224-295e-4e76-a27d-af37d2bec623 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.211142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1db652-27f2-420b-ae7a-769e4a0ca2cc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-98f07224-295e-4e76-a27d-af37d2bec623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:15.211324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:15.212375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 292/584] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:15.218296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f07224-295e-4e76-a27d-af37d2bec623 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.229086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:15.256895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84875cdd-42bf-4115-b962-c1df8c68f763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.257835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84875cdd-42bf-4115-b962-c1df8c68f763 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.258729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84875cdd-42bf-4115-b962-c1df8c68f763 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.259134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84875cdd-42bf-4115-b962-c1df8c68f763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.268240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:15.273639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84875cdd-42bf-4115-b962-c1df8c68f763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.275501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84875cdd-42bf-4115-b962-c1df8c68f763 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-84875cdd-42bf-4115-b962-c1df8c68f763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:15.294984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:15.295882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:15.301549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:15.301549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:15.301549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:15.301549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:15.301549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 293/585] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:15.301549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84875cdd-42bf-4115-b962-c1df8c68f763 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.331732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f07224-295e-4e76-a27d-af37d2bec623 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.366938 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c1126-fd9a-4df1-8716-1c913d7c6ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.367734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c1126-fd9a-4df1-8716-1c913d7c6ee9 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.368019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c1126-fd9a-4df1-8716-1c913d7c6ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.368295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c1126-fd9a-4df1-8716-1c913d7c6ee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.379058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c1126-fd9a-4df1-8716-1c913d7c6ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.380520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c1126-fd9a-4df1-8716-1c913d7c6ee9 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-a99c1126-fd9a-4df1-8716-1c913d7c6ee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:15.397725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:15.398647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:15.399516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:15.399516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:15.399516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:15.399516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:15.399516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 293/586] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:15.399798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c1126-fd9a-4df1-8716-1c913d7c6ee9 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.412864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84875cdd-42bf-4115-b962-c1df8c68f763 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.450081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f866ac8-51ec-4b4e-8859-d17f8e25c058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.450881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f866ac8-51ec-4b4e-8859-d17f8e25c058 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.451319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f866ac8-51ec-4b4e-8859-d17f8e25c058 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.451430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f866ac8-51ec-4b4e-8859-d17f8e25c058 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.458951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f866ac8-51ec-4b4e-8859-d17f8e25c058 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.460388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f866ac8-51ec-4b4e-8859-d17f8e25c058 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-4f866ac8-51ec-4b4e-8859-d17f8e25c058 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:15.474905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:15.475854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 294/587] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:15.481641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f866ac8-51ec-4b4e-8859-d17f8e25c058 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.506271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:15.509161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c1126-fd9a-4df1-8716-1c913d7c6ee9 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.509423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:15.543259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7141a9c-34c6-4439-88b2-7de1bef9f039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.545271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7141a9c-34c6-4439-88b2-7de1bef9f039 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.545502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7141a9c-34c6-4439-88b2-7de1bef9f039 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.545778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7141a9c-34c6-4439-88b2-7de1bef9f039 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.546727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7141a9c-34c6-4439-88b2-7de1bef9f039 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:15.552172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7141a9c-34c6-4439-88b2-7de1bef9f039 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.552172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7141a9c-34c6-4439-88b2-7de1bef9f039 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.560690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 294/588] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:15.561508 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7141a9c-34c6-4439-88b2-7de1bef9f039 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.593988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f866ac8-51ec-4b4e-8859-d17f8e25c058 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.630398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6cdc8-63fb-4c68-be56-36ba6814a9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.631336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6cdc8-63fb-4c68-be56-36ba6814a9fe None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.632195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6cdc8-63fb-4c68-be56-36ba6814a9fe None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.632757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6cdc8-63fb-4c68-be56-36ba6814a9fe None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.633840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6cdc8-63fb-4c68-be56-36ba6814a9fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:15.634334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6cdc8-63fb-4c68-be56-36ba6814a9fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.635892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6cdc8-63fb-4c68-be56-36ba6814a9fe None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.649352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 295/589] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:15.650498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6cdc8-63fb-4c68-be56-36ba6814a9fe None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.672997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7141a9c-34c6-4439-88b2-7de1bef9f039 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.711290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcd193-2191-41c9-94dd-9ab47c07094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.711965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd193-2191-41c9-94dd-9ab47c07094e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.712251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd193-2191-41c9-94dd-9ab47c07094e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.712526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcd193-2191-41c9-94dd-9ab47c07094e None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.713252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbcd193-2191-41c9-94dd-9ab47c07094e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:15.713556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbcd193-2191-41c9-94dd-9ab47c07094e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.714969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbcd193-2191-41c9-94dd-9ab47c07094e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.727068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 295/590] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:15.727859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcd193-2191-41c9-94dd-9ab47c07094e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.747483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:15.750041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:15.768551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6cdc8-63fb-4c68-be56-36ba6814a9fe None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.805931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a69e49-d338-4b2b-9aa7-37278cc02277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.805931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a69e49-d338-4b2b-9aa7-37278cc02277 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.805931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a69e49-d338-4b2b-9aa7-37278cc02277 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.805931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a69e49-d338-4b2b-9aa7-37278cc02277 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.806360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a69e49-d338-4b2b-9aa7-37278cc02277 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:15.806360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a69e49-d338-4b2b-9aa7-37278cc02277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.807255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a69e49-d338-4b2b-9aa7-37278cc02277 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.824940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 296/591] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:15.825747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a69e49-d338-4b2b-9aa7-37278cc02277 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:15.846293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcd193-2191-41c9-94dd-9ab47c07094e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.882155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88832c5-d6d2-4d20-8489-4c0df61a787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.883132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88832c5-d6d2-4d20-8489-4c0df61a787b None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.883709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88832c5-d6d2-4d20-8489-4c0df61a787b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.884042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88832c5-d6d2-4d20-8489-4c0df61a787b None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/ce88c973feef4627a1b871ee9cb26511/roles/3cb613c2ddca467295425cf3890d1dc3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.898930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88832c5-d6d2-4d20-8489-4c0df61a787b None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=ce88c973feef4627a1b871ee9cb26511, role_id=3cb613c2ddca467295425cf3890d1dc3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.900449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88832c5-d6d2-4d20-8489-4c0df61a787b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.932038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a69e49-d338-4b2b-9aa7-37278cc02277 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:15.965794 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb55a0e-1909-47df-afe0-b1f89c539f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:15.966637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55a0e-1909-47df-afe0-b1f89c539f80 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:15.966917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55a0e-1909-47df-afe0-b1f89c539f80 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:15.967191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55a0e-1909-47df-afe0-b1f89c539f80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:15.973811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb55a0e-1909-47df-afe0-b1f89c539f80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:15.975251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb55a0e-1909-47df-afe0-b1f89c539f80 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:15.977657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-fcb55a0e-1909-47df-afe0-b1f89c539f80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:15.989494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:15.990435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:15.991330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:15.991330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:15.991330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:15.991330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:15.991330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:15.991330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:15.991330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:15.991330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:15.991963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 297/592] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:15.992883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb55a0e-1909-47df-afe0-b1f89c539f80 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:16.003147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 296/593] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:35:15 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/ce88c973feef4627a1b871ee9cb26511/roles/3cb613c2ddca467295425cf3890d1dc3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:35:16.010827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88832c5-d6d2-4d20-8489-4c0df61a787b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:16.139315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb55a0e-1909-47df-afe0-b1f89c539f80 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:16.174995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:16.186635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc752-df6c-4a85-a453-c8973458a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:16.187499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc752-df6c-4a85-a453-c8973458a6bc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.187787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc752-df6c-4a85-a453-c8973458a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.188064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc752-df6c-4a85-a453-c8973458a6bc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:16.198170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cc752-df6c-4a85-a453-c8973458a6bc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:16.199618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cc752-df6c-4a85-a453-c8973458a6bc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-8a9cc752-df6c-4a85-a453-c8973458a6bc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:16.217896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:16.218942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:35:16.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:16.220392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88832c5-d6d2-4d20-8489-4c0df61a787b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:16.220467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 298/594] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:16.220467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc752-df6c-4a85-a453-c8973458a6bc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:16.258583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd776b0-f349-448e-9e6b-19274ef667a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:16.259547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd776b0-f349-448e-9e6b-19274ef667a0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.259935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd776b0-f349-448e-9e6b-19274ef667a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.260339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd776b0-f349-448e-9e6b-19274ef667a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:16.271399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd776b0-f349-448e-9e6b-19274ef667a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:16.273046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd776b0-f349-448e-9e6b-19274ef667a0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:16.290161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-6bd776b0-f349-448e-9e6b-19274ef667a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:16.292503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:16.293578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:16.294504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:16.294504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:16.294504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:16.294504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:16.295056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 297/595] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:16.295927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd776b0-f349-448e-9e6b-19274ef667a0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:16.342252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc752-df6c-4a85-a453-c8973458a6bc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:16.389127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc8216-0bd0-4d78-9c55-4e501cefa14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:16.390448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc8216-0bd0-4d78-9c55-4e501cefa14f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.390757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc8216-0bd0-4d78-9c55-4e501cefa14f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.391112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc8216-0bd0-4d78-9c55-4e501cefa14f None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:16.391994 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc8216-0bd0-4d78-9c55-4e501cefa14f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:16.392788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc8216-0bd0-4d78-9c55-4e501cefa14f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:16.394803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc8216-0bd0-4d78-9c55-4e501cefa14f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:16.407935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 299/596] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:16.413461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc8216-0bd0-4d78-9c55-4e501cefa14f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:16.423623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd776b0-f349-448e-9e6b-19274ef667a0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:16.450697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:16.462756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04234ce4-22bd-4a07-b7f8-06b2bb1d3de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:16.463429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04234ce4-22bd-4a07-b7f8-06b2bb1d3de2 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.463710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04234ce4-22bd-4a07-b7f8-06b2bb1d3de2 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.463993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04234ce4-22bd-4a07-b7f8-06b2bb1d3de2 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:16.465547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04234ce4-22bd-4a07-b7f8-06b2bb1d3de2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:16.465547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04234ce4-22bd-4a07-b7f8-06b2bb1d3de2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:16.466905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04234ce4-22bd-4a07-b7f8-06b2bb1d3de2 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:16.486945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 298/597] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Jul 14 15:35:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:16.487892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04234ce4-22bd-4a07-b7f8-06b2bb1d3de2 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:16.542850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc8216-0bd0-4d78-9c55-4e501cefa14f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:16.570246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:16.592191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c1ef2-dccc-40a8-add8-72cd36736308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:16.593479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c1ef2-dccc-40a8-add8-72cd36736308 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.594221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c1ef2-dccc-40a8-add8-72cd36736308 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.598094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c1ef2-dccc-40a8-add8-72cd36736308 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:16.598094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c1ef2-dccc-40a8-add8-72cd36736308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:16.598094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c1ef2-dccc-40a8-add8-72cd36736308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:16.598094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c1ef2-dccc-40a8-add8-72cd36736308 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:16.611244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04234ce4-22bd-4a07-b7f8-06b2bb1d3de2 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:16.620023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 300/598] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:16.621004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c1ef2-dccc-40a8-add8-72cd36736308 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:16.650689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764e6d-3ccd-42a4-82e5-3c9b9ec185d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:16.652083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764e6d-3ccd-42a4-82e5-3c9b9ec185d8 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.652366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764e6d-3ccd-42a4-82e5-3c9b9ec185d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.652641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764e6d-3ccd-42a4-82e5-3c9b9ec185d8 None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/5eeda3264191409eba35ffbb908770a6/roles/3cb613c2ddca467295425cf3890d1dc3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:16.671044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764e6d-3ccd-42a4-82e5-3c9b9ec185d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=5eeda3264191409eba35ffbb908770a6, role_id=3cb613c2ddca467295425cf3890d1dc3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:16.673155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764e6d-3ccd-42a4-82e5-3c9b9ec185d8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:16.726245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c1ef2-dccc-40a8-add8-72cd36736308 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:16.759526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae576af2-ae36-47c0-8151-dedb7a3006b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:16.760336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae576af2-ae36-47c0-8151-dedb7a3006b2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.760620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae576af2-ae36-47c0-8151-dedb7a3006b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.760900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae576af2-ae36-47c0-8151-dedb7a3006b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:16.764854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 299/599] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:35:16 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/5eeda3264191409eba35ffbb908770a6/roles/3cb613c2ddca467295425cf3890d1dc3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:35:16.765660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764e6d-3ccd-42a4-82e5-3c9b9ec185d8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:16.786067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae576af2-ae36-47c0-8151-dedb7a3006b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:16.788789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae576af2-ae36-47c0-8151-dedb7a3006b2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:16.805587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-ae576af2-ae36-47c0-8151-dedb7a3006b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:16.815159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:16.816484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:16.817414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:16.817414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:16.817414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:16.817414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:16.821958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 301/600] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:16.823612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad07cfe-1bd8-4522-944a-f138050fc3dc None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.824593 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad07cfe-1bd8-4522-944a-f138050fc3dc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.825486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad07cfe-1bd8-4522-944a-f138050fc3dc None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:16.828751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 302/601] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:16.828751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad07cfe-1bd8-4522-944a-f138050fc3dc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:16.874750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:16.911930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764e6d-3ccd-42a4-82e5-3c9b9ec185d8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:16.954463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebf9ed-58ff-449b-91be-5f7d4c70f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:16.955272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebf9ed-58ff-449b-91be-5f7d4c70f154 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.955574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebf9ed-58ff-449b-91be-5f7d4c70f154 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.955836 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebf9ed-58ff-449b-91be-5f7d4c70f154 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:16.963408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebf9ed-58ff-449b-91be-5f7d4c70f154 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:16.964885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebf9ed-58ff-449b-91be-5f7d4c70f154 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:16.975520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad07cfe-1bd8-4522-944a-f138050fc3dc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-d8ebf9ed-58ff-449b-91be-5f7d4c70f154 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:16.980272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:16.981241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:16.982210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:16.982210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:16.982210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:16.982210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:16.982210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:16.982210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:16.982210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:16.982210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:16.985937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 300/602] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:16.987220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd61ed0-0883-48cf-a8f8-5805010891ac None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:16.987512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd61ed0-0883-48cf-a8f8-5805010891ac None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:16.987796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd61ed0-0883-48cf-a8f8-5805010891ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:17.005445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-4bd61ed0-0883-48cf-a8f8-5805010891ac None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:17.006666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4bd61ed0-0883-48cf-a8f8-5805010891ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:17.021963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3943c-da04-43e0-87cf-dfa47b97b270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:17.023053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3943c-da04-43e0-87cf-dfa47b97b270 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:17.023480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3943c-da04-43e0-87cf-dfa47b97b270 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:17.024089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3943c-da04-43e0-87cf-dfa47b97b270 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:17.025346 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3943c-da04-43e0-87cf-dfa47b97b270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:17.025665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3943c-da04-43e0-87cf-dfa47b97b270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:17.027133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3943c-da04-43e0-87cf-dfa47b97b270 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:17.046203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 303/603] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:17.047181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3943c-da04-43e0-87cf-dfa47b97b270 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:17.133957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 301/604] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:17.135134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd61ed0-0883-48cf-a8f8-5805010891ac None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:17.168958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:17.201631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:17.211658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3943c-da04-43e0-87cf-dfa47b97b270 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:17.250388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1289d75-7a24-408a-9574-982c45df2c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:17.251214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289d75-7a24-408a-9574-982c45df2c31 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:17.251602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289d75-7a24-408a-9574-982c45df2c31 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:17.251988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1289d75-7a24-408a-9574-982c45df2c31 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:17.252888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1289d75-7a24-408a-9574-982c45df2c31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:17.253306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1289d75-7a24-408a-9574-982c45df2c31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:17.254869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1289d75-7a24-408a-9574-982c45df2c31 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:17.264221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd61ed0-0883-48cf-a8f8-5805010891ac None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:17.268425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 304/605] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:17.274038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1289d75-7a24-408a-9574-982c45df2c31 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:17.310297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6fc07-8aa5-4639-b17f-d0eeeb2a31e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:17.312096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6fc07-8aa5-4639-b17f-d0eeeb2a31e0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:17.312387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6fc07-8aa5-4639-b17f-d0eeeb2a31e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:17.312670 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6fc07-8aa5-4639-b17f-d0eeeb2a31e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:17.322030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb6fc07-8aa5-4639-b17f-d0eeeb2a31e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:17.324324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb6fc07-8aa5-4639-b17f-d0eeeb2a31e0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-5fb6fc07-8aa5-4639-b17f-d0eeeb2a31e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:17.344270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:17.345352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:17.346340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:17.346340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:17.346340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:17.346340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:17.346340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:17.346340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:17.346340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:17.346340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:17.346772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 302/606] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:17.347697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6fc07-8aa5-4639-b17f-d0eeeb2a31e0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:17.385474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1289d75-7a24-408a-9574-982c45df2c31 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:17.421504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a341954-7752-4dba-a11d-77e8c2c4c2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:17.422408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a341954-7752-4dba-a11d-77e8c2c4c2bb None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:17.422789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a341954-7752-4dba-a11d-77e8c2c4c2bb None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:17.423167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a341954-7752-4dba-a11d-77e8c2c4c2bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:17.434601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a341954-7752-4dba-a11d-77e8c2c4c2bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-7a341954-7752-4dba-a11d-77e8c2c4c2bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:17.442656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:17.443665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:17.444499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:17.445128 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 305/607] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:17.445878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a341954-7752-4dba-a11d-77e8c2c4c2bb None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:17.461123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:17.462241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:17.487605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6fc07-8aa5-4639-b17f-d0eeeb2a31e0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:17.533260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707bb30-c69c-4d11-a26b-5e526a1551b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:17.535044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707bb30-c69c-4d11-a26b-5e526a1551b0 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:17.535485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707bb30-c69c-4d11-a26b-5e526a1551b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:17.535921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707bb30-c69c-4d11-a26b-5e526a1551b0 None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/3d54d684efce44b99f3235e80beb1c56/roles/3cb613c2ddca467295425cf3890d1dc3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:17.553215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a341954-7752-4dba-a11d-77e8c2c4c2bb None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:17.560652 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707bb30-c69c-4d11-a26b-5e526a1551b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=3d54d684efce44b99f3235e80beb1c56, role_id=3cb613c2ddca467295425cf3890d1dc3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:17.562956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707bb30-c69c-4d11-a26b-5e526a1551b0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:17.591733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce1103-860e-4855-a5a3-ca652cbe81b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:17.592452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce1103-860e-4855-a5a3-ca652cbe81b5 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:17.592744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce1103-860e-4855-a5a3-ca652cbe81b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:17.593032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce1103-860e-4855-a5a3-ca652cbe81b5 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:17.594559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce1103-860e-4855-a5a3-ca652cbe81b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:17.594559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce1103-860e-4855-a5a3-ca652cbe81b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:17.595722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce1103-860e-4855-a5a3-ca652cbe81b5 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:17.609674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 306/608] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:17.610304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce1103-860e-4855-a5a3-ca652cbe81b5 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:17.666249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 303/609] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:35:17 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/3d54d684efce44b99f3235e80beb1c56/roles/3cb613c2ddca467295425cf3890d1dc3 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:35:17.667714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707bb30-c69c-4d11-a26b-5e526a1551b0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:17.695958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:17.774760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce1103-860e-4855-a5a3-ca652cbe81b5 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:17.801004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707bb30-c69c-4d11-a26b-5e526a1551b0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:17.838085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:17.846717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e75c5-188e-459e-bfa0-6c896474de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:17.847978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e75c5-188e-459e-bfa0-6c896474de85 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:17.848437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e75c5-188e-459e-bfa0-6c896474de85 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:17.850811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e75c5-188e-459e-bfa0-6c896474de85 None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/7e63b8e907764f2c818391ab2ad9184c/roles/3cb613c2ddca467295425cf3890d1dc3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:17.868112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e75c5-188e-459e-bfa0-6c896474de85 None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=7e63b8e907764f2c818391ab2ad9184c, role_id=3cb613c2ddca467295425cf3890d1dc3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:17.870625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e75c5-188e-459e-bfa0-6c896474de85 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:17.870935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176ca0a-74e7-450d-95a9-e68c9d5ecd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:17.871694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176ca0a-74e7-450d-95a9-e68c9d5ecd43 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:17.871983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176ca0a-74e7-450d-95a9-e68c9d5ecd43 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:17.872260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176ca0a-74e7-450d-95a9-e68c9d5ecd43 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:17.873037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176ca0a-74e7-450d-95a9-e68c9d5ecd43 None admin] RBAC: Adding query filter params (name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:17.873342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176ca0a-74e7-450d-95a9-e68c9d5ecd43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:17.874781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176ca0a-74e7-450d-95a9-e68c9d5ecd43 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:17.890903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 307/610] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:17.896433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176ca0a-74e7-450d-95a9-e68c9d5ecd43 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:17.942077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 304/611] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:35:17 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/7e63b8e907764f2c818391ab2ad9184c/roles/3cb613c2ddca467295425cf3890d1dc3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:35:18.021835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176ca0a-74e7-450d-95a9-e68c9d5ecd43 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:18.073146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022f352-dd40-4212-a584-f1a445a25bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:18.074291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022f352-dd40-4212-a584-f1a445a25bc0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:18.075261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022f352-dd40-4212-a584-f1a445a25bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:18.075709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022f352-dd40-4212-a584-f1a445a25bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:18.078411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:18.088790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022f352-dd40-4212-a584-f1a445a25bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:18.090709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022f352-dd40-4212-a584-f1a445a25bc0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-f022f352-dd40-4212-a584-f1a445a25bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:18.110395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:18.111382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:18.112311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:18.112311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:18.112311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:18.112311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:18.117914 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 308/612] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:18.120420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e75c5-188e-459e-bfa0-6c896474de85 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:18.232253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e75c5-188e-459e-bfa0-6c896474de85 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:18.268514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e63475-b47a-449e-9d4f-522d6830d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:18.269475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e63475-b47a-449e-9d4f-522d6830d57c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:18.269959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e63475-b47a-449e-9d4f-522d6830d57c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:18.270383 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e63475-b47a-449e-9d4f-522d6830d57c None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:18.271262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e63475-b47a-449e-9d4f-522d6830d57c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:18.271736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e63475-b47a-449e-9d4f-522d6830d57c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:18.273318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e63475-b47a-449e-9d4f-522d6830d57c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:18.289965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 305/613] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:18.292523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022f352-dd40-4212-a584-f1a445a25bc0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:18.333861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c9825-24b6-4dd6-9bfa-6850c5589d10 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:18.334167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c9825-24b6-4dd6-9bfa-6850c5589d10 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:18.334433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c9825-24b6-4dd6-9bfa-6850c5589d10 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:18.335956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 306/614] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:18.342925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5927f-952d-471e-aa98-d67c2bbb545f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:18.343206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5927f-952d-471e-aa98-d67c2bbb545f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:18.343490 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5927f-952d-471e-aa98-d67c2bbb545f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:18.363352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-7fe5927f-952d-471e-aa98-d67c2bbb545f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:18.364699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7fe5927f-952d-471e-aa98-d67c2bbb545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:18.401839 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:18.414924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022f352-dd40-4212-a584-f1a445a25bc0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:18.437020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 307/615] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:35:18.455736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877e1c5-0a45-406d-897c-e43a28bad4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:18.456597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877e1c5-0a45-406d-897c-e43a28bad4d3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:18.456888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877e1c5-0a45-406d-897c-e43a28bad4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:18.457169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877e1c5-0a45-406d-897c-e43a28bad4d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:18.464961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5927f-952d-471e-aa98-d67c2bbb545f None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:18.471696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877e1c5-0a45-406d-897c-e43a28bad4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:18.473539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4877e1c5-0a45-406d-897c-e43a28bad4d3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-4877e1c5-0a45-406d-897c-e43a28bad4d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:18.499599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:18.500621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:18.501565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:18.510027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 309/616] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:18.524124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877e1c5-0a45-406d-897c-e43a28bad4d3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:18.597193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5927f-952d-471e-aa98-d67c2bbb545f None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:18.626632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:18.656958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f0e0a-295c-492b-ae51-3d1b1c362999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:18.658342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0e0a-295c-492b-ae51-3d1b1c362999 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:18.658830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0e0a-295c-492b-ae51-3d1b1c362999 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:18.659118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f0e0a-295c-492b-ae51-3d1b1c362999 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:18.671852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f0e0a-295c-492b-ae51-3d1b1c362999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-ba4f0e0a-295c-492b-ae51-3d1b1c362999 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:18.680217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:18.681172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:18.682037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 308/617] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:18.689694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f0e0a-295c-492b-ae51-3d1b1c362999 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:18.730153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:18.763450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877e1c5-0a45-406d-897c-e43a28bad4d3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:18.835233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701edde9-7594-42b4-b5d3-a3d71ed40f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:18.836408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701edde9-7594-42b4-b5d3-a3d71ed40f6b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:18.836694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701edde9-7594-42b4-b5d3-a3d71ed40f6b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:18.836955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701edde9-7594-42b4-b5d3-a3d71ed40f6b None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:18.837811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701edde9-7594-42b4-b5d3-a3d71ed40f6b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:18.838143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701edde9-7594-42b4-b5d3-a3d71ed40f6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:18.839597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701edde9-7594-42b4-b5d3-a3d71ed40f6b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:18.871565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f0e0a-295c-492b-ae51-3d1b1c362999 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:18.886456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 310/618] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 14 15:35:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:18.888547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701edde9-7594-42b4-b5d3-a3d71ed40f6b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:18.947085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad74432-8e64-4592-bf5d-00ca05078b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:18.947769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad74432-8e64-4592-bf5d-00ca05078b7a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:18.948058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad74432-8e64-4592-bf5d-00ca05078b7a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:18.948336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad74432-8e64-4592-bf5d-00ca05078b7a None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:18.949095 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad74432-8e64-4592-bf5d-00ca05078b7a None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:18.949403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad74432-8e64-4592-bf5d-00ca05078b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:18.950884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad74432-8e64-4592-bf5d-00ca05078b7a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:18.969783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 309/619] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:18.975328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad74432-8e64-4592-bf5d-00ca05078b7a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:18.997940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701edde9-7594-42b4-b5d3-a3d71ed40f6b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.000333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:19.041162 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dc991-3531-45d7-99b0-75ff6036b48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.042118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dc991-3531-45d7-99b0-75ff6036b48e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.042404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dc991-3531-45d7-99b0-75ff6036b48e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.042727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dc991-3531-45d7-99b0-75ff6036b48e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.055189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dc991-3531-45d7-99b0-75ff6036b48e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:19.056574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dc991-3531-45d7-99b0-75ff6036b48e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-520dc991-3531-45d7-99b0-75ff6036b48e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:19.077363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:19.078602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:19.079818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:19.079818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:19.079818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:19.079818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:19.082017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 311/620] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:19.083558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeb2b6-c8f7-4fd0-bb44-6edd68145e47 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.083853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeb2b6-c8f7-4fd0-bb44-6edd68145e47 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.084136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeb2b6-c8f7-4fd0-bb44-6edd68145e47 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.089644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 312/621] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:19.089644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeb2b6-c8f7-4fd0-bb44-6edd68145e47 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:19.115280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:19.126115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad74432-8e64-4592-bf5d-00ca05078b7a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.162239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66d237-173d-4c67-9477-c8c4a27ec24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.163369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66d237-173d-4c67-9477-c8c4a27ec24e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.163853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66d237-173d-4c67-9477-c8c4a27ec24e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.164322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66d237-173d-4c67-9477-c8c4a27ec24e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.175569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66d237-173d-4c67-9477-c8c4a27ec24e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:19.177268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66d237-173d-4c67-9477-c8c4a27ec24e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:19.194633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeb2b6-c8f7-4fd0-bb44-6edd68145e47 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-8e66d237-173d-4c67-9477-c8c4a27ec24e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:19.196545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:19.197415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:19.198363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:19.198363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:19.198363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:19.198363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:19.204838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 310/622] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:19.206386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7ec4d-2e73-4583-be2a-bb96fb03f66f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.206953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7ec4d-2e73-4583-be2a-bb96fb03f66f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.207429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7ec4d-2e73-4583-be2a-bb96fb03f66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.224954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5db7ec4d-2e73-4583-be2a-bb96fb03f66f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:19.226421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5db7ec4d-2e73-4583-be2a-bb96fb03f66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:19.231339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d572a-9be7-4ce8-b739-17722a9790ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.232060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d572a-9be7-4ce8-b739-17722a9790ca None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.232344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d572a-9be7-4ce8-b739-17722a9790ca None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.232624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d572a-9be7-4ce8-b739-17722a9790ca None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.233412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d572a-9be7-4ce8-b739-17722a9790ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:19.233726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d572a-9be7-4ce8-b739-17722a9790ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:19.235239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d572a-9be7-4ce8-b739-17722a9790ca None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:19.252360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 313/623] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:19.253274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d572a-9be7-4ce8-b739-17722a9790ca None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:19.286216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:19.300671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 311/624] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:19.301781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7ec4d-2e73-4583-be2a-bb96fb03f66f None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:19.358413 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d572a-9be7-4ce8-b739-17722a9790ca None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.385556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:19.395014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d210d-358d-4235-97a9-8f70944d5884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.396538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d210d-358d-4235-97a9-8f70944d5884 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.396538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d210d-358d-4235-97a9-8f70944d5884 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.396538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d210d-358d-4235-97a9-8f70944d5884 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.397009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d210d-358d-4235-97a9-8f70944d5884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:19.397326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d210d-358d-4235-97a9-8f70944d5884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:19.398742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d210d-358d-4235-97a9-8f70944d5884 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:19.415917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 314/625] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:19.415917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d210d-358d-4235-97a9-8f70944d5884 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:19.419470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7ec4d-2e73-4583-be2a-bb96fb03f66f None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.462142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114654ca-1f26-449b-9f0d-a9731a791588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.463000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114654ca-1f26-449b-9f0d-a9731a791588 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.463285 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114654ca-1f26-449b-9f0d-a9731a791588 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.463562 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114654ca-1f26-449b-9f0d-a9731a791588 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.475749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114654ca-1f26-449b-9f0d-a9731a791588 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:19.478988 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114654ca-1f26-449b-9f0d-a9731a791588 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-114654ca-1f26-449b-9f0d-a9731a791588 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:19.500584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:19.501572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:19.502513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:19.502513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:19.502513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:19.502513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:19.502513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:19.502513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:19.502513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:19.502513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:19.504066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 312/626] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:19.505148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114654ca-1f26-449b-9f0d-a9731a791588 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:19.520563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:19.544166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d210d-358d-4235-97a9-8f70944d5884 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.582215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5f74f-c779-4852-85a6-ecba7fb68204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.583072 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5f74f-c779-4852-85a6-ecba7fb68204 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.583359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5f74f-c779-4852-85a6-ecba7fb68204 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.583686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5f74f-c779-4852-85a6-ecba7fb68204 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.595164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5f74f-c779-4852-85a6-ecba7fb68204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-eee5f74f-c779-4852-85a6-ecba7fb68204 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:19.603211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:19.604183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:19.605026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 315/627] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:19.606269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5f74f-c779-4852-85a6-ecba7fb68204 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:19.652474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114654ca-1f26-449b-9f0d-a9731a791588 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.699825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb3380-855f-4b7e-9428-8cbc0a403bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.701230 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb3380-855f-4b7e-9428-8cbc0a403bab None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.702072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb3380-855f-4b7e-9428-8cbc0a403bab None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.702923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb3380-855f-4b7e-9428-8cbc0a403bab None admin] PATH_INFO: `/v3/users/nova` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.705007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:19.714718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb3380-855f-4b7e-9428-8cbc0a403bab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:19.719458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb3380-855f-4b7e-9428-8cbc0a403bab None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:19.719562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5f74f-c779-4852-85a6-ecba7fb68204 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-83bb3380-855f-4b7e-9428-8cbc0a403bab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:19.733493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:19.734533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:19.735485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:19.739369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 313/628] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 14 15:35:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:19.740948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb3380-855f-4b7e-9428-8cbc0a403bab None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:19.768690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36e775-fa42-4f56-a803-982a43139dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.769949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36e775-fa42-4f56-a803-982a43139dd4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.770376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36e775-fa42-4f56-a803-982a43139dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.771047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36e775-fa42-4f56-a803-982a43139dd4 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.772412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36e775-fa42-4f56-a803-982a43139dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:19.773133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36e775-fa42-4f56-a803-982a43139dd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:19.775392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36e775-fa42-4f56-a803-982a43139dd4 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:19.778664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:19.792431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 316/629] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:19.793458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36e775-fa42-4f56-a803-982a43139dd4 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:19.866150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb3380-855f-4b7e-9428-8cbc0a403bab None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.905459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d872f23-8ff2-46df-ada8-655d5fa80d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.906260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d872f23-8ff2-46df-ada8-655d5fa80d4b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.906668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d872f23-8ff2-46df-ada8-655d5fa80d4b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.907031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d872f23-8ff2-46df-ada8-655d5fa80d4b None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.907886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d872f23-8ff2-46df-ada8-655d5fa80d4b None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:19.908315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d872f23-8ff2-46df-ada8-655d5fa80d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:19.909799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d872f23-8ff2-46df-ada8-655d5fa80d4b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:19.919194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 314/630] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:19.919900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36e775-fa42-4f56-a803-982a43139dd4 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:19.920527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d872f23-8ff2-46df-ada8-655d5fa80d4b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:19.964636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16e76a-b660-4d89-851b-5babb3b63df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:19.966121 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16e76a-b660-4d89-851b-5babb3b63df2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:19.966547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16e76a-b660-4d89-851b-5babb3b63df2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:19.966961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16e76a-b660-4d89-851b-5babb3b63df2 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:19.967944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16e76a-b660-4d89-851b-5babb3b63df2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:19.968420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16e76a-b660-4d89-851b-5babb3b63df2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:19.970567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16e76a-b660-4d89-851b-5babb3b63df2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:19.987609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:19.992015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 317/631] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 14 15:35:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:19.993066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16e76a-b660-4d89-851b-5babb3b63df2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:20.039696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d872f23-8ff2-46df-ada8-655d5fa80d4b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:20.058421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:20.080607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1319d-5708-445c-876b-fc0de2c0bb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:20.081638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1319d-5708-445c-876b-fc0de2c0bb82 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.082026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1319d-5708-445c-876b-fc0de2c0bb82 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.082311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1319d-5708-445c-876b-fc0de2c0bb82 None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/95f36b4517ee4b64bc43e000c854df30/roles/3cb613c2ddca467295425cf3890d1dc3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.098655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1319d-5708-445c-876b-fc0de2c0bb82 None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=95f36b4517ee4b64bc43e000c854df30, role_id=3cb613c2ddca467295425cf3890d1dc3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:20.100210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1319d-5708-445c-876b-fc0de2c0bb82 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:20.105594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16e76a-b660-4d89-851b-5babb3b63df2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:20.140904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e6b26-b01a-4024-b43a-52ff6cf31600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:20.141701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e6b26-b01a-4024-b43a-52ff6cf31600 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.142016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e6b26-b01a-4024-b43a-52ff6cf31600 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.142300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e6b26-b01a-4024-b43a-52ff6cf31600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.160443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 315/632] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:35:19 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/95f36b4517ee4b64bc43e000c854df30/roles/3cb613c2ddca467295425cf3890d1dc3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:35:20.160443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94274cf1-52fc-4757-aaf3-4337abc740df None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.160443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94274cf1-52fc-4757-aaf3-4337abc740df None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.160443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94274cf1-52fc-4757-aaf3-4337abc740df None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.160443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 316/633] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:20.160443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94274cf1-52fc-4757-aaf3-4337abc740df None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:20.163926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e6b26-b01a-4024-b43a-52ff6cf31600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:20.165549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e6b26-b01a-4024-b43a-52ff6cf31600 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-227e6b26-b01a-4024-b43a-52ff6cf31600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:20.187367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:20.190284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:20.191479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:20.191479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:20.191479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:20.191479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:20.193589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 318/634] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:20.195152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53648a2-19d0-4827-9998-4daac4f525ce None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.195445 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53648a2-19d0-4827-9998-4daac4f525ce None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.195730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53648a2-19d0-4827-9998-4daac4f525ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.213253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e53648a2-19d0-4827-9998-4daac4f525ce None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:20.214607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e53648a2-19d0-4827-9998-4daac4f525ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:20.298232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 319/635] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:20.299314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53648a2-19d0-4827-9998-4daac4f525ce None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:20.309125 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94274cf1-52fc-4757-aaf3-4337abc740df None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:20.324276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:20.358923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:20.370440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a56d4-9c34-4423-a5bd-8a2d9b958043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:20.371186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a56d4-9c34-4423-a5bd-8a2d9b958043 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.371529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a56d4-9c34-4423-a5bd-8a2d9b958043 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.371808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a56d4-9c34-4423-a5bd-8a2d9b958043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.430668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53648a2-19d0-4827-9998-4daac4f525ce None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:20.453363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a56d4-9c34-4423-a5bd-8a2d9b958043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:20.454869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a56d4-9c34-4423-a5bd-8a2d9b958043 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:20.466600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e3891-3164-44d5-be75-118a4ce2d550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:20.467560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e3891-3164-44d5-be75-118a4ce2d550 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.468045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e3891-3164-44d5-be75-118a4ce2d550 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.468515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e3891-3164-44d5-be75-118a4ce2d550 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.469444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e3891-3164-44d5-be75-118a4ce2d550 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:20.469977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e3891-3164-44d5-be75-118a4ce2d550 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:20.471597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e3891-3164-44d5-be75-118a4ce2d550 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:20.487821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 320/636] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:20.488958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e3891-3164-44d5-be75-118a4ce2d550 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:20.555045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 317/637] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:20.557044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a6eb7-9b75-4cf9-80dc-4d1150f328df None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.557505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a6eb7-9b75-4cf9-80dc-4d1150f328df None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.557989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a6eb7-9b75-4cf9-80dc-4d1150f328df None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.564019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 318/638] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:20.564830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a6eb7-9b75-4cf9-80dc-4d1150f328df None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:20.598337 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e3891-3164-44d5-be75-118a4ce2d550 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:20.633262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86e41e-aa9b-4ac3-885c-86772b3dd522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:20.634206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86e41e-aa9b-4ac3-885c-86772b3dd522 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.634498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86e41e-aa9b-4ac3-885c-86772b3dd522 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.634775 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86e41e-aa9b-4ac3-885c-86772b3dd522 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.645801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86e41e-aa9b-4ac3-885c-86772b3dd522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-4a86e41e-aa9b-4ac3-885c-86772b3dd522 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:20.654232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:20.655301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:20.656203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 321/639] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:20.660897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86e41e-aa9b-4ac3-885c-86772b3dd522 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:20.670198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a6eb7-9b75-4cf9-80dc-4d1150f328df None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:20.694206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:20.707308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3234687a-61c4-413b-89ad-f88bcd6d213a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:20.708111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234687a-61c4-413b-89ad-f88bcd6d213a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.708397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234687a-61c4-413b-89ad-f88bcd6d213a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.708679 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234687a-61c4-413b-89ad-f88bcd6d213a None admin] PATH_INFO: `/v3/users/glance` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.716726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3234687a-61c4-413b-89ad-f88bcd6d213a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:20.718206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3234687a-61c4-413b-89ad-f88bcd6d213a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:20.723416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-3234687a-61c4-413b-89ad-f88bcd6d213a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:20.735709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:20.736767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:20.737711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 319/640] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:20.738919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997ddb9-abf1-4faf-914b-e4f3ae8f496a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.738919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997ddb9-abf1-4faf-914b-e4f3ae8f496a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.739099 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997ddb9-abf1-4faf-914b-e4f3ae8f496a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.755787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5997ddb9-abf1-4faf-914b-e4f3ae8f496a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:20.756970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5997ddb9-abf1-4faf-914b-e4f3ae8f496a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:20.771612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86e41e-aa9b-4ac3-885c-86772b3dd522 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:20.806334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685accc-6ace-4957-9628-24d19bc3aebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:20.807132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685accc-6ace-4957-9628-24d19bc3aebe None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.807417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685accc-6ace-4957-9628-24d19bc3aebe None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.807692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685accc-6ace-4957-9628-24d19bc3aebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.818607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:20.818899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685accc-6ace-4957-9628-24d19bc3aebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:20.820475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685accc-6ace-4957-9628-24d19bc3aebe None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:20.828782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 320/641] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:20.848495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997ddb9-abf1-4faf-914b-e4f3ae8f496a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-a685accc-6ace-4957-9628-24d19bc3aebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:20.848622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:20.849556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:20.850507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:20.850507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:20.850507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:20.850507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:20.854316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 322/642] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:20.855203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685accc-6ace-4957-9628-24d19bc3aebe None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:20.866848 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:20.938677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997ddb9-abf1-4faf-914b-e4f3ae8f496a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:20.960904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685accc-6ace-4957-9628-24d19bc3aebe None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:20.976502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed39dc1-280b-458a-aea6-f61e72fb99d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:20.977299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39dc1-280b-458a-aea6-f61e72fb99d0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:20.977685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39dc1-280b-458a-aea6-f61e72fb99d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:20.978167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed39dc1-280b-458a-aea6-f61e72fb99d0 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:20.978995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed39dc1-280b-458a-aea6-f61e72fb99d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:20.979526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed39dc1-280b-458a-aea6-f61e72fb99d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:20.981417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed39dc1-280b-458a-aea6-f61e72fb99d0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:20.994789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 321/643] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:20.994789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed39dc1-280b-458a-aea6-f61e72fb99d0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.001408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bd27f-4ec7-4c40-a35f-e2d949a2c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.002203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bd27f-4ec7-4c40-a35f-e2d949a2c4ca None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.002557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bd27f-4ec7-4c40-a35f-e2d949a2c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.002904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bd27f-4ec7-4c40-a35f-e2d949a2c4ca None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.003818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bd27f-4ec7-4c40-a35f-e2d949a2c4ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:21.004203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bd27f-4ec7-4c40-a35f-e2d949a2c4ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:21.005634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bd27f-4ec7-4c40-a35f-e2d949a2c4ca None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:21.023082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 323/644] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 14 15:35:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:21.023888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bd27f-4ec7-4c40-a35f-e2d949a2c4ca None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:21.098777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:21.111903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed39dc1-280b-458a-aea6-f61e72fb99d0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.138940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bd27f-4ec7-4c40-a35f-e2d949a2c4ca None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.147719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55db81b-b2de-4f7d-8a46-8cbf1a7aaf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.148381 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55db81b-b2de-4f7d-8a46-8cbf1a7aaf48 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.148794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55db81b-b2de-4f7d-8a46-8cbf1a7aaf48 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.148949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55db81b-b2de-4f7d-8a46-8cbf1a7aaf48 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.149739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55db81b-b2de-4f7d-8a46-8cbf1a7aaf48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:21.150081 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55db81b-b2de-4f7d-8a46-8cbf1a7aaf48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:21.151526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55db81b-b2de-4f7d-8a46-8cbf1a7aaf48 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:21.164031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 322/645] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:21.164952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55db81b-b2de-4f7d-8a46-8cbf1a7aaf48 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.180130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d5958-f67f-42ec-b4b3-bc41054973b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.181059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d5958-f67f-42ec-b4b3-bc41054973b7 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.181451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d5958-f67f-42ec-b4b3-bc41054973b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.181808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d5958-f67f-42ec-b4b3-bc41054973b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.194444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d5958-f67f-42ec-b4b3-bc41054973b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-ac0d5958-f67f-42ec-b4b3-bc41054973b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:21.202970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:21.204131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 324/646] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:21.205316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d5958-f67f-42ec-b4b3-bc41054973b7 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.274811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55db81b-b2de-4f7d-8a46-8cbf1a7aaf48 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.293558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:21.310995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b079b5-fca1-4908-af7a-58e1db9c7b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.311809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b079b5-fca1-4908-af7a-58e1db9c7b3f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.312100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b079b5-fca1-4908-af7a-58e1db9c7b3f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.312383 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b079b5-fca1-4908-af7a-58e1db9c7b3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.317296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d5958-f67f-42ec-b4b3-bc41054973b7 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.324788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b079b5-fca1-4908-af7a-58e1db9c7b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:21.326330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b079b5-fca1-4908-af7a-58e1db9c7b3f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-b8b079b5-fca1-4908-af7a-58e1db9c7b3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:21.345346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:21.346371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:21.347266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:21.347266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:21.347266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:21.347266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:21.347266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 323/647] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:21.347560 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b079b5-fca1-4908-af7a-58e1db9c7b3f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.353248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:21.363242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854627de-82f8-403d-ba08-e05aca7c650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.371854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854627de-82f8-403d-ba08-e05aca7c650f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.371854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854627de-82f8-403d-ba08-e05aca7c650f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.371854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854627de-82f8-403d-ba08-e05aca7c650f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.438943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854627de-82f8-403d-ba08-e05aca7c650f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:21.441036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854627de-82f8-403d-ba08-e05aca7c650f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:21.455845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b079b5-fca1-4908-af7a-58e1db9c7b3f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.492060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d433d-227b-452a-ad20-ed4afdddf7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.492852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d433d-227b-452a-ad20-ed4afdddf7fb None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.493137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d433d-227b-452a-ad20-ed4afdddf7fb None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.493413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d433d-227b-452a-ad20-ed4afdddf7fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.500807 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d433d-227b-452a-ad20-ed4afdddf7fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:21.502298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d433d-227b-452a-ad20-ed4afdddf7fb None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-c84d433d-227b-452a-ad20-ed4afdddf7fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:21.516201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:21.517133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:21.518061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:21.518061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:21.518061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:21.518061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:21.518061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:21.518061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:21.518061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:21.518061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:21.518061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 324/648] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:21.519080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d433d-227b-452a-ad20-ed4afdddf7fb None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.551783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 325/649] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:21.552574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854627de-82f8-403d-ba08-e05aca7c650f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.556249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:21.580921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:21.629486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d433d-227b-452a-ad20-ed4afdddf7fb None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.658250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854627de-82f8-403d-ba08-e05aca7c650f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.668659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a90a-5221-43c5-ae87-3792367bd042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.669344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a90a-5221-43c5-ae87-3792367bd042 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.669625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a90a-5221-43c5-ae87-3792367bd042 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.669935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164a90a-5221-43c5-ae87-3792367bd042 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.670693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164a90a-5221-43c5-ae87-3792367bd042 None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:21.671002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164a90a-5221-43c5-ae87-3792367bd042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:21.672422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164a90a-5221-43c5-ae87-3792367bd042 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:21.681867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 325/650] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:21.681867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a90a-5221-43c5-ae87-3792367bd042 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.700677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38cf31-5c13-442f-93b2-fdc9ea17f310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.701406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38cf31-5c13-442f-93b2-fdc9ea17f310 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.701692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38cf31-5c13-442f-93b2-fdc9ea17f310 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.702002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38cf31-5c13-442f-93b2-fdc9ea17f310 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.702734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38cf31-5c13-442f-93b2-fdc9ea17f310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:21.703047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38cf31-5c13-442f-93b2-fdc9ea17f310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:21.704406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38cf31-5c13-442f-93b2-fdc9ea17f310 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:21.720642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 326/651] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:21.720642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38cf31-5c13-442f-93b2-fdc9ea17f310 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.777923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:21.786776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164a90a-5221-43c5-ae87-3792367bd042 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.813149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:21.823435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dcbe3-3f47-46f7-b0ac-34bd1c411c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.824290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dcbe3-3f47-46f7-b0ac-34bd1c411c5e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.824389 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dcbe3-3f47-46f7-b0ac-34bd1c411c5e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.825175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dcbe3-3f47-46f7-b0ac-34bd1c411c5e None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.825998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dcbe3-3f47-46f7-b0ac-34bd1c411c5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:21.826311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dcbe3-3f47-46f7-b0ac-34bd1c411c5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:21.827751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dcbe3-3f47-46f7-b0ac-34bd1c411c5e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:21.828760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38cf31-5c13-442f-93b2-fdc9ea17f310 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.845267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 326/652] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:21.845267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dcbe3-3f47-46f7-b0ac-34bd1c411c5e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.869534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc3a6e-2a78-493e-ae12-622c89c1a5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.870375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc3a6e-2a78-493e-ae12-622c89c1a5b5 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.870652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc3a6e-2a78-493e-ae12-622c89c1a5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.870928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc3a6e-2a78-493e-ae12-622c89c1a5b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:21.881687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc3a6e-2a78-493e-ae12-622c89c1a5b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:21.883146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc3a6e-2a78-493e-ae12-622c89c1a5b5 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-99fc3a6e-2a78-493e-ae12-622c89c1a5b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:21.900704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:21.901674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:21.902644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:21.902644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:21.902644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:21.902644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:21.902644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 327/653] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:21.902957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc3a6e-2a78-493e-ae12-622c89c1a5b5 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:21.954179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dcbe3-3f47-46f7-b0ac-34bd1c411c5e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:21.990162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799630f-a82b-45bb-a41d-779d28556aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:21.990963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799630f-a82b-45bb-a41d-779d28556aba None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:21.991254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799630f-a82b-45bb-a41d-779d28556aba None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:21.991529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799630f-a82b-45bb-a41d-779d28556aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.002659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799630f-a82b-45bb-a41d-779d28556aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.004124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799630f-a82b-45bb-a41d-779d28556aba None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.008527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc3a6e-2a78-493e-ae12-622c89c1a5b5 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-6799630f-a82b-45bb-a41d-779d28556aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:22.022973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:22.023905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:22.024781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:22.024781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:22.024781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:22.024781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:22.024781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 327/654] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:22.025065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799630f-a82b-45bb-a41d-779d28556aba None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:22.032851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:22.053163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35813e-b738-4c51-a366-e865e7f3395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:22.054057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35813e-b738-4c51-a366-e865e7f3395f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.054587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35813e-b738-4c51-a366-e865e7f3395f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.054587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35813e-b738-4c51-a366-e865e7f3395f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.063020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35813e-b738-4c51-a366-e865e7f3395f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.064509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35813e-b738-4c51-a366-e865e7f3395f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.066772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-3d35813e-b738-4c51-a366-e865e7f3395f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:22.082730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:22.083699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:35:22.084639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:22.088649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 328/655] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:22.089393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35813e-b738-4c51-a366-e865e7f3395f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:22.139925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799630f-a82b-45bb-a41d-779d28556aba None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:22.176424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7395bc3-6fab-49d6-9e8b-b36fd20e97b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:22.177094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7395bc3-6fab-49d6-9e8b-b36fd20e97b0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.177380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7395bc3-6fab-49d6-9e8b-b36fd20e97b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.177659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7395bc3-6fab-49d6-9e8b-b36fd20e97b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.178526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7395bc3-6fab-49d6-9e8b-b36fd20e97b0 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=ce88c973feef4627a1b871ee9cb26511) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:22.178842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7395bc3-6fab-49d6-9e8b-b36fd20e97b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.180287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7395bc3-6fab-49d6-9e8b-b36fd20e97b0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.186960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 328/656] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/role_assignments?user.id=ce88c973feef4627a1b871ee9cb26511&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:22.187748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7395bc3-6fab-49d6-9e8b-b36fd20e97b0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:22.214582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35813e-b738-4c51-a366-e865e7f3395f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:22.255103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e2d8-7594-43cb-a590-2db65ca4120f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:22.255802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e2d8-7594-43cb-a590-2db65ca4120f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.256088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e2d8-7594-43cb-a590-2db65ca4120f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.256366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12e2d8-7594-43cb-a590-2db65ca4120f None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.257109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e2d8-7594-43cb-a590-2db65ca4120f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:22.257418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e2d8-7594-43cb-a590-2db65ca4120f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.258860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12e2d8-7594-43cb-a590-2db65ca4120f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.273875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 329/657] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:22.274082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e2d8-7594-43cb-a590-2db65ca4120f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:22.285095 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:22.322411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7395bc3-6fab-49d6-9e8b-b36fd20e97b0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:22.343685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:22.362341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5a6dc-9009-4dfa-93d0-46d68dfb04e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:22.363020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5a6dc-9009-4dfa-93d0-46d68dfb04e8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.363304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5a6dc-9009-4dfa-93d0-46d68dfb04e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.368275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5a6dc-9009-4dfa-93d0-46d68dfb04e8 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.368275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5a6dc-9009-4dfa-93d0-46d68dfb04e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:22.368275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5a6dc-9009-4dfa-93d0-46d68dfb04e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.368275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5a6dc-9009-4dfa-93d0-46d68dfb04e8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.381969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 329/658] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:22.383591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b07994-be5e-463b-9010-8e9bc9d341d0 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.383591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b07994-be5e-463b-9010-8e9bc9d341d0 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.383847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b07994-be5e-463b-9010-8e9bc9d341d0 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.385250 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 330/659] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:22.385883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b07994-be5e-463b-9010-8e9bc9d341d0 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:22.390601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12e2d8-7594-43cb-a590-2db65ca4120f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:22.430228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91c0c7-7d2a-4432-9d64-507235146176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:22.431001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91c0c7-7d2a-4432-9d64-507235146176 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.431278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91c0c7-7d2a-4432-9d64-507235146176 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.431553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91c0c7-7d2a-4432-9d64-507235146176 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.432399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91c0c7-7d2a-4432-9d64-507235146176 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:22.432706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91c0c7-7d2a-4432-9d64-507235146176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.434831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91c0c7-7d2a-4432-9d64-507235146176 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.453587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 330/660] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:22.454868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91c0c7-7d2a-4432-9d64-507235146176 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:22.503496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b07994-be5e-463b-9010-8e9bc9d341d0 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:22.535911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:22.544989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89dbbc-053e-40cc-b65f-0e843af8644b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:22.545803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89dbbc-053e-40cc-b65f-0e843af8644b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.546117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89dbbc-053e-40cc-b65f-0e843af8644b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.546396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89dbbc-053e-40cc-b65f-0e843af8644b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.553321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89dbbc-053e-40cc-b65f-0e843af8644b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.554812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89dbbc-053e-40cc-b65f-0e843af8644b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.560635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91c0c7-7d2a-4432-9d64-507235146176 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-de89dbbc-053e-40cc-b65f-0e843af8644b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:22.569697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:22.571164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:22.572264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:22.572264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:22.572264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:22.572264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:22.572264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:22.572264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:22.572264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:22.572264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:22.575147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 331/661] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:22.576343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d1cc0-4d1f-49be-b251-c4a0d2acb221 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.577063 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d1cc0-4d1f-49be-b251-c4a0d2acb221 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.577398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d1cc0-4d1f-49be-b251-c4a0d2acb221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.586563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:22.594303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-890d1cc0-4d1f-49be-b251-c4a0d2acb221 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:22.595517 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-890d1cc0-4d1f-49be-b251-c4a0d2acb221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:22.606785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70682d3-cc13-45b3-a049-36e777e4cf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:22.607992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70682d3-cc13-45b3-a049-36e777e4cf6d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.608281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70682d3-cc13-45b3-a049-36e777e4cf6d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.608562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70682d3-cc13-45b3-a049-36e777e4cf6d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.615957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70682d3-cc13-45b3-a049-36e777e4cf6d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.617412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70682d3-cc13-45b3-a049-36e777e4cf6d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-b70682d3-cc13-45b3-a049-36e777e4cf6d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:22.633505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:22.634772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 331/662] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:22.635812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70682d3-cc13-45b3-a049-36e777e4cf6d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:22.677850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 332/663] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:22.678729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d1cc0-4d1f-49be-b251-c4a0d2acb221 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:22.761173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70682d3-cc13-45b3-a049-36e777e4cf6d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:22.797132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d1cc0-4d1f-49be-b251-c4a0d2acb221 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:22.806558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29bf71-947a-418c-95f2-e071ded12fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:22.807227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29bf71-947a-418c-95f2-e071ded12fe7 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.807515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29bf71-947a-418c-95f2-e071ded12fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.807795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29bf71-947a-418c-95f2-e071ded12fe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.818378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:22.836376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681693e7-da31-4bfa-8298-3413cb4f05fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:22.836376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681693e7-da31-4bfa-8298-3413cb4f05fa None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:22.836778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681693e7-da31-4bfa-8298-3413cb4f05fa None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:22.836778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681693e7-da31-4bfa-8298-3413cb4f05fa None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:22.837566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681693e7-da31-4bfa-8298-3413cb4f05fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:22.837913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681693e7-da31-4bfa-8298-3413cb4f05fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.839331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681693e7-da31-4bfa-8298-3413cb4f05fa None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.852557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 333/664] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:22.858099 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681693e7-da31-4bfa-8298-3413cb4f05fa None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:22.864639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:22.888160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29bf71-947a-418c-95f2-e071ded12fe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:22.889611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29bf71-947a-418c-95f2-e071ded12fe7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:22.964533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681693e7-da31-4bfa-8298-3413cb4f05fa None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.004252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 332/665] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:23.004252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29bf71-947a-418c-95f2-e071ded12fe7 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:23.007203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de64ed9-1493-4158-89e8-9ed9f9ca57b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:23.007888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de64ed9-1493-4158-89e8-9ed9f9ca57b8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:23.008181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de64ed9-1493-4158-89e8-9ed9f9ca57b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:23.008465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de64ed9-1493-4158-89e8-9ed9f9ca57b8 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:23.009371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de64ed9-1493-4158-89e8-9ed9f9ca57b8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:23.009684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de64ed9-1493-4158-89e8-9ed9f9ca57b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:23.011580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de64ed9-1493-4158-89e8-9ed9f9ca57b8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:23.036143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 334/666] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:23.036143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de64ed9-1493-4158-89e8-9ed9f9ca57b8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:23.077981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:23.089204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:23.113405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29bf71-947a-418c-95f2-e071ded12fe7 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.145169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de64ed9-1493-4158-89e8-9ed9f9ca57b8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39b4be-ddb6-4ec3-ac3f-747ef3580516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:23.148236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39b4be-ddb6-4ec3-ac3f-747ef3580516 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:23.148523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39b4be-ddb6-4ec3-ac3f-747ef3580516 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:23.148805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39b4be-ddb6-4ec3-ac3f-747ef3580516 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:23.158998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39b4be-ddb6-4ec3-ac3f-747ef3580516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-ba39b4be-ddb6-4ec3-ac3f-747ef3580516 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:23.166699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:23.167678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:23.169074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 333/667] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:23.170166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39b4be-ddb6-4ec3-ac3f-747ef3580516 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:23.185655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35252bd4-f625-4d7c-bd95-77cb89d23f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:23.186339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35252bd4-f625-4d7c-bd95-77cb89d23f35 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:23.186623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35252bd4-f625-4d7c-bd95-77cb89d23f35 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:23.186901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35252bd4-f625-4d7c-bd95-77cb89d23f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:23.187736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35252bd4-f625-4d7c-bd95-77cb89d23f35 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=5eeda3264191409eba35ffbb908770a6) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:23.188047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35252bd4-f625-4d7c-bd95-77cb89d23f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:23.189479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35252bd4-f625-4d7c-bd95-77cb89d23f35 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:23.195696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 335/668] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:23 2022] GET /identity/v3/role_assignments?user.id=5eeda3264191409eba35ffbb908770a6&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:23.202903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35252bd4-f625-4d7c-bd95-77cb89d23f35 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:23.285236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39b4be-ddb6-4ec3-ac3f-747ef3580516 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.324980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb7adf-6381-4564-8054-b3fd0a584a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:23.325531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb7adf-6381-4564-8054-b3fd0a584a7d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:23.325831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb7adf-6381-4564-8054-b3fd0a584a7d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:23.326138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb7adf-6381-4564-8054-b3fd0a584a7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:23.329719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:23.351401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb7adf-6381-4564-8054-b3fd0a584a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:23.351401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb7adf-6381-4564-8054-b3fd0a584a7d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:23.357241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35252bd4-f625-4d7c-bd95-77cb89d23f35 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.359262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-a8fb7adf-6381-4564-8054-b3fd0a584a7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:23.361986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:23.363205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:23.364154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:23.364154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:23.364154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:23.364154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:23.367818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 334/669] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:23.368786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb7adf-6381-4564-8054-b3fd0a584a7d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:23.404742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094079-10f2-4c36-8a94-79ffcfb86aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:23.405450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094079-10f2-4c36-8a94-79ffcfb86aa2 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:23.405695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094079-10f2-4c36-8a94-79ffcfb86aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:23.406011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094079-10f2-4c36-8a94-79ffcfb86aa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:23.497223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de094079-10f2-4c36-8a94-79ffcfb86aa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:23.499053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de094079-10f2-4c36-8a94-79ffcfb86aa2 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:23.504455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb7adf-6381-4564-8054-b3fd0a584a7d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.553245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504770e4-379c-4f2c-a6aa-11ff92e09b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:23.554190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504770e4-379c-4f2c-a6aa-11ff92e09b37 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:23.554484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504770e4-379c-4f2c-a6aa-11ff92e09b37 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:23.554760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504770e4-379c-4f2c-a6aa-11ff92e09b37 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:23.555536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504770e4-379c-4f2c-a6aa-11ff92e09b37 None admin] RBAC: Adding query filter params (name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:23.556209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504770e4-379c-4f2c-a6aa-11ff92e09b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:23.557578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504770e4-379c-4f2c-a6aa-11ff92e09b37 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:23.570517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 335/670] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:23.575254 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504770e4-379c-4f2c-a6aa-11ff92e09b37 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:23.617808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:23.623364 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 336/671] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:23.624522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094079-10f2-4c36-8a94-79ffcfb86aa2 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:23.643105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:23.707581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504770e4-379c-4f2c-a6aa-11ff92e09b37 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.737543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094079-10f2-4c36-8a94-79ffcfb86aa2 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.751152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3660-8534-4672-a9cf-85090a2c9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:23.751922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3660-8534-4672-a9cf-85090a2c9537 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:23.752275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3660-8534-4672-a9cf-85090a2c9537 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:23.752627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3660-8534-4672-a9cf-85090a2c9537 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:23.753616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf3660-8534-4672-a9cf-85090a2c9537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:23.754014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf3660-8534-4672-a9cf-85090a2c9537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:23.756110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf3660-8534-4672-a9cf-85090a2c9537 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:23.776208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 336/672] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:23.776208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3660-8534-4672-a9cf-85090a2c9537 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:23.780694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83a42f-3183-4424-845d-95cd3941c157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:23.782542 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83a42f-3183-4424-845d-95cd3941c157 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:23.786186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83a42f-3183-4424-845d-95cd3941c157 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:23.786570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83a42f-3183-4424-845d-95cd3941c157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:23.801789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83a42f-3183-4424-845d-95cd3941c157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:23.804233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83a42f-3183-4424-845d-95cd3941c157 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-dd83a42f-3183-4424-845d-95cd3941c157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:23.828377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:23.829568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:23.830495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:23.830495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:23.830495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:23.830495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:23.830751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 337/673] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:23.835630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83a42f-3183-4424-845d-95cd3941c157 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:23.893654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:23.913643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3660-8534-4672-a9cf-85090a2c9537 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.929412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:23.956508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac91b3-6c5c-416c-b44a-e863103b1ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:23.957628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83a42f-3183-4424-845d-95cd3941c157 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:23.957922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac91b3-6c5c-416c-b44a-e863103b1ac0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:23.958399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac91b3-6c5c-416c-b44a-e863103b1ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:23.958872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac91b3-6c5c-416c-b44a-e863103b1ac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:23.973291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac91b3-6c5c-416c-b44a-e863103b1ac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:23.975101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac91b3-6c5c-416c-b44a-e863103b1ac0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-a9ac91b3-6c5c-416c-b44a-e863103b1ac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:23.994586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:23.995524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:23.996486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:23.996486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:23.996486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:23.996486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:23.999019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 337/674] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:24.000060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac91b3-6c5c-416c-b44a-e863103b1ac0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.006115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8998a-bcfc-4d73-99cd-f74381531228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.007041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8998a-bcfc-4d73-99cd-f74381531228 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.007393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8998a-bcfc-4d73-99cd-f74381531228 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.007740 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8998a-bcfc-4d73-99cd-f74381531228 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.017592 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8998a-bcfc-4d73-99cd-f74381531228 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:24.019930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8998a-bcfc-4d73-99cd-f74381531228 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-f5e8998a-bcfc-4d73-99cd-f74381531228 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:24.044669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:24.045648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:24.046593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:24.046593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:24.046593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:24.046593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:24.046593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:24.046593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:24.046593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:24.046593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:24.049896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 338/675] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:24.057855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8998a-bcfc-4d73-99cd-f74381531228 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.122397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac91b3-6c5c-416c-b44a-e863103b1ac0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:24.158659 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d757aa1-8738-47fa-a600-b2870cc76bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.159366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d757aa1-8738-47fa-a600-b2870cc76bd9 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.159719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d757aa1-8738-47fa-a600-b2870cc76bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.160065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d757aa1-8738-47fa-a600-b2870cc76bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.160868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d757aa1-8738-47fa-a600-b2870cc76bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:24.161244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d757aa1-8738-47fa-a600-b2870cc76bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:24.162725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d757aa1-8738-47fa-a600-b2870cc76bd9 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:24.180112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 338/676] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:24.181341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d757aa1-8738-47fa-a600-b2870cc76bd9 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.188523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8998a-bcfc-4d73-99cd-f74381531228 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:24.206516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:24.210742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:24.228367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2a591-1375-41e2-ad13-0240d67eac70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.229037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2a591-1375-41e2-ad13-0240d67eac70 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.229376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2a591-1375-41e2-ad13-0240d67eac70 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.229660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2a591-1375-41e2-ad13-0240d67eac70 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.230435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2a591-1375-41e2-ad13-0240d67eac70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:24.230861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2a591-1375-41e2-ad13-0240d67eac70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:24.232238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2a591-1375-41e2-ad13-0240d67eac70 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:24.250025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 339/677] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:24.250025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2a591-1375-41e2-ad13-0240d67eac70 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.307656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d757aa1-8738-47fa-a600-b2870cc76bd9 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:24.349098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f39013-4b66-43ab-96e4-6f4fe85b4296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.349799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f39013-4b66-43ab-96e4-6f4fe85b4296 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.350109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f39013-4b66-43ab-96e4-6f4fe85b4296 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.350604 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f39013-4b66-43ab-96e4-6f4fe85b4296 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.351162 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f39013-4b66-43ab-96e4-6f4fe85b4296 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:24.351549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f39013-4b66-43ab-96e4-6f4fe85b4296 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:24.352911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f39013-4b66-43ab-96e4-6f4fe85b4296 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:24.360332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2a591-1375-41e2-ad13-0240d67eac70 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:24.370999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 339/678] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:24.370999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f39013-4b66-43ab-96e4-6f4fe85b4296 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.395585 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb55a04-3c1b-4bd6-99a4-676855c66f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.396605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb55a04-3c1b-4bd6-99a4-676855c66f59 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.396896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb55a04-3c1b-4bd6-99a4-676855c66f59 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.397166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb55a04-3c1b-4bd6-99a4-676855c66f59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.405868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb55a04-3c1b-4bd6-99a4-676855c66f59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:24.407482 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb55a04-3c1b-4bd6-99a4-676855c66f59 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-9bb55a04-3c1b-4bd6-99a4-676855c66f59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:24.423689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:24.424687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:24.425647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 340/679] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:24.426766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a313a-5012-4998-8fc2-3472b88270fc None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.426766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a313a-5012-4998-8fc2-3472b88270fc None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.427064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a313a-5012-4998-8fc2-3472b88270fc None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.432634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 341/680] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:24.432634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a313a-5012-4998-8fc2-3472b88270fc None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.463724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:24.468981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:24.487358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f39013-4b66-43ab-96e4-6f4fe85b4296 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:24.525983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290827-dd53-4320-a79c-d5e8d38953e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.527239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290827-dd53-4320-a79c-d5e8d38953e6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.528171 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290827-dd53-4320-a79c-d5e8d38953e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.528450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290827-dd53-4320-a79c-d5e8d38953e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.536382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a313a-5012-4998-8fc2-3472b88270fc None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:24.536822 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64290827-dd53-4320-a79c-d5e8d38953e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:24.543145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64290827-dd53-4320-a79c-d5e8d38953e6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-64290827-dd53-4320-a79c-d5e8d38953e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:24.556136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:24.557043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:24.558059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:24.558059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:24.558059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:24.558059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:24.558059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:24.558059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:24.558059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:24.558059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:24.561965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 340/681] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:24.562853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290827-dd53-4320-a79c-d5e8d38953e6 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.576233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b9cec-f943-4cfb-af07-a2fa3fe9d963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.576966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b9cec-f943-4cfb-af07-a2fa3fe9d963 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.577272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b9cec-f943-4cfb-af07-a2fa3fe9d963 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.577459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b9cec-f943-4cfb-af07-a2fa3fe9d963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.578300 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b9cec-f943-4cfb-af07-a2fa3fe9d963 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=3d54d684efce44b99f3235e80beb1c56) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:24.578710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b9cec-f943-4cfb-af07-a2fa3fe9d963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:24.580044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b9cec-f943-4cfb-af07-a2fa3fe9d963 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:24.593864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 342/682] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/role_assignments?user.id=3d54d684efce44b99f3235e80beb1c56&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:24.593864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4677be3-5cc1-4103-8e3f-4b63b2f4af0b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.593864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4677be3-5cc1-4103-8e3f-4b63b2f4af0b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.593864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4677be3-5cc1-4103-8e3f-4b63b2f4af0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.617281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f4677be3-5cc1-4103-8e3f-4b63b2f4af0b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:24.619511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f4677be3-5cc1-4103-8e3f-4b63b2f4af0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:24.692278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290827-dd53-4320-a79c-d5e8d38953e6 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:24.714733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 343/683] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:24.716050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4677be3-5cc1-4103-8e3f-4b63b2f4af0b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.728452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397382e8-dec5-46cd-81b8-848637023d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.729140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397382e8-dec5-46cd-81b8-848637023d5a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.729422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397382e8-dec5-46cd-81b8-848637023d5a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.729698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397382e8-dec5-46cd-81b8-848637023d5a None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.730608 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397382e8-dec5-46cd-81b8-848637023d5a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:24.731444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397382e8-dec5-46cd-81b8-848637023d5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:24.732900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397382e8-dec5-46cd-81b8-848637023d5a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:24.748568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:24.750309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:24.753307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 341/684] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:24.758362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397382e8-dec5-46cd-81b8-848637023d5a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.836461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4677be3-5cc1-4103-8e3f-4b63b2f4af0b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:24.876304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397382e8-dec5-46cd-81b8-848637023d5a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:24.889806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4630937-d8e7-41cf-bf43-1d79de262a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.890510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630937-d8e7-41cf-bf43-1d79de262a63 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.890797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630937-d8e7-41cf-bf43-1d79de262a63 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.891087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4630937-d8e7-41cf-bf43-1d79de262a63 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.892121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630937-d8e7-41cf-bf43-1d79de262a63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:24.892242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630937-d8e7-41cf-bf43-1d79de262a63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:24.893593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4630937-d8e7-41cf-bf43-1d79de262a63 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:24.907196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 344/685] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:24.908054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4630937-d8e7-41cf-bf43-1d79de262a63 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:24.922338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b21d7-0326-4505-af68-f2c53aa5710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:24.923266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b21d7-0326-4505-af68-f2c53aa5710f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:24.923561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b21d7-0326-4505-af68-f2c53aa5710f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:24.923844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b21d7-0326-4505-af68-f2c53aa5710f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:24.939803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b21d7-0326-4505-af68-f2c53aa5710f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-689b21d7-0326-4505-af68-f2c53aa5710f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:24.950154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:24.951312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:24.952821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 342/686] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:24.954478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b21d7-0326-4505-af68-f2c53aa5710f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:25.012795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:25.029165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4630937-d8e7-41cf-bf43-1d79de262a63 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:25.042389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:25.081579 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee58fe-96ab-4e3c-bd28-36ef44386c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:25.082789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee58fe-96ab-4e3c-bd28-36ef44386c75 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.083663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee58fe-96ab-4e3c-bd28-36ef44386c75 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.084373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee58fe-96ab-4e3c-bd28-36ef44386c75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.092190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b21d7-0326-4505-af68-f2c53aa5710f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:25.134895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee7651-635a-4312-ba35-c27649d1d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:25.136332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee7651-635a-4312-ba35-c27649d1d227 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.136770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee7651-635a-4312-ba35-c27649d1d227 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.137746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee7651-635a-4312-ba35-c27649d1d227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.139494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee7651-635a-4312-ba35-c27649d1d227 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=7e63b8e907764f2c818391ab2ad9184c) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:25.139975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee7651-635a-4312-ba35-c27649d1d227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:25.142100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee7651-635a-4312-ba35-c27649d1d227 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:25.151555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 343/687] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/role_assignments?user.id=7e63b8e907764f2c818391ab2ad9184c&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:25.152619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee7651-635a-4312-ba35-c27649d1d227 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:25.170542 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee58fe-96ab-4e3c-bd28-36ef44386c75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:25.172016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee58fe-96ab-4e3c-bd28-36ef44386c75 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:25.274770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee7651-635a-4312-ba35-c27649d1d227 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:25.302760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:25.315783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276819dd-0f1e-413c-bf07-a43247338160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:25.316767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276819dd-0f1e-413c-bf07-a43247338160 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.317203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276819dd-0f1e-413c-bf07-a43247338160 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.317636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276819dd-0f1e-413c-bf07-a43247338160 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.318686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276819dd-0f1e-413c-bf07-a43247338160 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:25.319151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276819dd-0f1e-413c-bf07-a43247338160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:25.320724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276819dd-0f1e-413c-bf07-a43247338160 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:25.331953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 344/688] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:25.337809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:25.339418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276819dd-0f1e-413c-bf07-a43247338160 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:25.375805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 345/689] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:25.381352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee58fe-96ab-4e3c-bd28-36ef44386c75 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:25.451869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276819dd-0f1e-413c-bf07-a43247338160 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:25.495235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611c620-b8c0-4dc7-9f09-6b6d8bb51b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:25.497250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611c620-b8c0-4dc7-9f09-6b6d8bb51b0f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.497693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611c620-b8c0-4dc7-9f09-6b6d8bb51b0f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.498155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611c620-b8c0-4dc7-9f09-6b6d8bb51b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.511884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611c620-b8c0-4dc7-9f09-6b6d8bb51b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:25.513646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611c620-b8c0-4dc7-9f09-6b6d8bb51b0f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:25.529550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee58fe-96ab-4e3c-bd28-36ef44386c75 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-f611c620-b8c0-4dc7-9f09-6b6d8bb51b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:25.538085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:25.539101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:25.539991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:25.539991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:25.539991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:25.539991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:25.539991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 345/690] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:25.540715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e22e44-3e9a-46f9-9b18-08364687f3e0 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.541228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e22e44-3e9a-46f9-9b18-08364687f3e0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.541333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e22e44-3e9a-46f9-9b18-08364687f3e0 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.542837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 346/691] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:25.545583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e22e44-3e9a-46f9-9b18-08364687f3e0 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:25.571784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3ef06-7d7c-4fe2-bf85-17f1bcd2b56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:25.572595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3ef06-7d7c-4fe2-bf85-17f1bcd2b56c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.572883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3ef06-7d7c-4fe2-bf85-17f1bcd2b56c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.573161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3ef06-7d7c-4fe2-bf85-17f1bcd2b56c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.585723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3ef06-7d7c-4fe2-bf85-17f1bcd2b56c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:25.587351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3ef06-7d7c-4fe2-bf85-17f1bcd2b56c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:25.593931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-32a3ef06-7d7c-4fe2-bf85-17f1bcd2b56c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:25.607141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:25.613617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:25.614613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:25.614613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:25.614613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:25.614613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:25.614613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 346/692] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:25.614613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62afa9c-a40d-474a-89ed-aad920be5536 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.614613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62afa9c-a40d-474a-89ed-aad920be5536 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.614613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62afa9c-a40d-474a-89ed-aad920be5536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.629331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e62afa9c-a40d-474a-89ed-aad920be5536 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:25.630870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e62afa9c-a40d-474a-89ed-aad920be5536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:25.663554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:25.676310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e22e44-3e9a-46f9-9b18-08364687f3e0 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:25.721840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 347/693] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:25.721840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62afa9c-a40d-474a-89ed-aad920be5536 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:25.729860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8c728-83c1-4ec6-b150-8fb7ada49317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:25.730629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8c728-83c1-4ec6-b150-8fb7ada49317 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.732228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8c728-83c1-4ec6-b150-8fb7ada49317 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.732697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8c728-83c1-4ec6-b150-8fb7ada49317 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.733441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8c728-83c1-4ec6-b150-8fb7ada49317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:25.734206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8c728-83c1-4ec6-b150-8fb7ada49317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:25.737351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8c728-83c1-4ec6-b150-8fb7ada49317 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:25.755082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 347/694] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:25.755939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8c728-83c1-4ec6-b150-8fb7ada49317 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:25.836956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62afa9c-a40d-474a-89ed-aad920be5536 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:25.868218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:25.873537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe64f90-d648-4a3a-90fc-b196c061802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:25.874504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe64f90-d648-4a3a-90fc-b196c061802d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.874619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe64f90-d648-4a3a-90fc-b196c061802d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.874921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe64f90-d648-4a3a-90fc-b196c061802d None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.875195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8c728-83c1-4ec6-b150-8fb7ada49317 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:25.875793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe64f90-d648-4a3a-90fc-b196c061802d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:25.876162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe64f90-d648-4a3a-90fc-b196c061802d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:25.877674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe64f90-d648-4a3a-90fc-b196c061802d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:25.891395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 348/695] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:25.892410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe64f90-d648-4a3a-90fc-b196c061802d None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:25.915026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84254f-1d30-4b80-83de-ee1eb489d34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:25.916218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84254f-1d30-4b80-83de-ee1eb489d34b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:25.916728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84254f-1d30-4b80-83de-ee1eb489d34b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:25.917224 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84254f-1d30-4b80-83de-ee1eb489d34b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:25.922618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:25.930977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84254f-1d30-4b80-83de-ee1eb489d34b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:25.932740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84254f-1d30-4b80-83de-ee1eb489d34b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-bc84254f-1d30-4b80-83de-ee1eb489d34b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:25.952241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:25.953333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:25.954270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:25.954270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:25.954270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:25.954270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:25.954621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 348/696] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:25.955768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84254f-1d30-4b80-83de-ee1eb489d34b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:26.003806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe64f90-d648-4a3a-90fc-b196c061802d None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:26.052628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5359c-572e-48f4-9f2c-c5ff23c10354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:26.053440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5359c-572e-48f4-9f2c-c5ff23c10354 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.053720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5359c-572e-48f4-9f2c-c5ff23c10354 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.054030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef5359c-572e-48f4-9f2c-c5ff23c10354 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.063086 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5359c-572e-48f4-9f2c-c5ff23c10354 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:26.065007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef5359c-572e-48f4-9f2c-c5ff23c10354 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:26.077615 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84254f-1d30-4b80-83de-ee1eb489d34b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-6ef5359c-572e-48f4-9f2c-c5ff23c10354 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:26.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:26.082702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:26.083875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:26.087234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 349/697] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 14 15:35:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:26.087234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5359c-572e-48f4-9f2c-c5ff23c10354 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:26.133114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa7d94-cc2c-4e38-b6cb-6e1455cbb2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:26.134104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa7d94-cc2c-4e38-b6cb-6e1455cbb2f7 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.134467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa7d94-cc2c-4e38-b6cb-6e1455cbb2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.134821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa7d94-cc2c-4e38-b6cb-6e1455cbb2f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.144654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa7d94-cc2c-4e38-b6cb-6e1455cbb2f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:26.146735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa7d94-cc2c-4e38-b6cb-6e1455cbb2f7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-38aa7d94-cc2c-4e38-b6cb-6e1455cbb2f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:26.166618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:26.167735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:26.169691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:26.169691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:26.169691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:26.169691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:26.169691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:26.169691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:26.169691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:26.169691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:26.170602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 349/698] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:26.175137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa7d94-cc2c-4e38-b6cb-6e1455cbb2f7 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:26.188086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:26.191451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:26.203582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef5359c-572e-48f4-9f2c-c5ff23c10354 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:26.238969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a66ec-58c8-48a0-83ab-a582a4d73f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:26.239745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a66ec-58c8-48a0-83ab-a582a4d73f46 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.240091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a66ec-58c8-48a0-83ab-a582a4d73f46 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.240438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a66ec-58c8-48a0-83ab-a582a4d73f46 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.241266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a66ec-58c8-48a0-83ab-a582a4d73f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:26.241667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a66ec-58c8-48a0-83ab-a582a4d73f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:26.243173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a66ec-58c8-48a0-83ab-a582a4d73f46 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:26.261179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 350/699] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:26.262131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a66ec-58c8-48a0-83ab-a582a4d73f46 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:26.298260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa7d94-cc2c-4e38-b6cb-6e1455cbb2f7 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:26.346447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013479b1-628b-44a7-81e5-313d121a0c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:26.348184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013479b1-628b-44a7-81e5-313d121a0c0d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.348628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013479b1-628b-44a7-81e5-313d121a0c0d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.349057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013479b1-628b-44a7-81e5-313d121a0c0d None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.350094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013479b1-628b-44a7-81e5-313d121a0c0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:26.350556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013479b1-628b-44a7-81e5-313d121a0c0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:26.352650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013479b1-628b-44a7-81e5-313d121a0c0d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:26.378819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 350/700] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 14 15:35:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:26.379703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013479b1-628b-44a7-81e5-313d121a0c0d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:26.385064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a66ec-58c8-48a0-83ab-a582a4d73f46 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:26.433146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb6cc6-3149-412d-ad73-d6e0d6222f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:26.434684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb6cc6-3149-412d-ad73-d6e0d6222f56 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.435127 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb6cc6-3149-412d-ad73-d6e0d6222f56 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.435856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb6cc6-3149-412d-ad73-d6e0d6222f56 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.437078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb6cc6-3149-412d-ad73-d6e0d6222f56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:26.437991 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb6cc6-3149-412d-ad73-d6e0d6222f56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:26.440118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb6cc6-3149-412d-ad73-d6e0d6222f56 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:26.449003 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:26.458535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 351/701] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:26.460061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb6cc6-3149-412d-ad73-d6e0d6222f56 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:26.480060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:26.494073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013479b1-628b-44a7-81e5-313d121a0c0d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:26.530288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a111360-f315-4713-9195-fa0102e006b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:26.531352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a111360-f315-4713-9195-fa0102e006b6 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.531647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a111360-f315-4713-9195-fa0102e006b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.531917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a111360-f315-4713-9195-fa0102e006b6 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.532771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a111360-f315-4713-9195-fa0102e006b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:26.534679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a111360-f315-4713-9195-fa0102e006b6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:26.592937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb6cc6-3149-412d-ad73-d6e0d6222f56 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:26.630874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 351/702] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:35:26 2022] POST /identity/v3/users => generated 254 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:26.630874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a111360-f315-4713-9195-fa0102e006b6 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:26.650701 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4519c1-e5b3-4f06-894c-5fdf977f23ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:26.651380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4519c1-e5b3-4f06-894c-5fdf977f23ed None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.651655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4519c1-e5b3-4f06-894c-5fdf977f23ed None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.651935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4519c1-e5b3-4f06-894c-5fdf977f23ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.761979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a111360-f315-4713-9195-fa0102e006b6 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:26.798241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:26.803482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896072e-17e1-47c3-ae27-35dea38e8454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:26.804734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896072e-17e1-47c3-ae27-35dea38e8454 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.805263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896072e-17e1-47c3-ae27-35dea38e8454 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.805768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896072e-17e1-47c3-ae27-35dea38e8454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.807632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896072e-17e1-47c3-ae27-35dea38e8454 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=95f36b4517ee4b64bc43e000c854df30) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:26.808175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896072e-17e1-47c3-ae27-35dea38e8454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:26.813970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896072e-17e1-47c3-ae27-35dea38e8454 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:26.817730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 352/703] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:26 2022] GET /identity/v3/role_assignments?user.id=95f36b4517ee4b64bc43e000c854df30&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:26.827103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d39f85-691a-4fb9-9e12-845818d56515 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.827760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d39f85-691a-4fb9-9e12-845818d56515 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.828274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d39f85-691a-4fb9-9e12-845818d56515 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.833964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 353/704] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:26.836542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edee733-5b88-47a6-acc1-667dd4c83c60 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:26.837150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edee733-5b88-47a6-acc1-667dd4c83c60 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:26.837633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edee733-5b88-47a6-acc1-667dd4c83c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:26.858747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-2edee733-5b88-47a6-acc1-667dd4c83c60 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:26.860702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2edee733-5b88-47a6-acc1-667dd4c83c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:26.864832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4519c1-e5b3-4f06-894c-5fdf977f23ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:26.866460 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4519c1-e5b3-4f06-894c-5fdf977f23ed None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:26.879409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:26.969926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 354/705] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:35:26.995260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 352/706] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:27.000301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edee733-5b88-47a6-acc1-667dd4c83c60 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:27.003241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4519c1-e5b3-4f06-894c-5fdf977f23ed None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:27.132866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4519c1-e5b3-4f06-894c-5fdf977f23ed None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:27.135300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edee733-5b88-47a6-acc1-667dd4c83c60 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:27.146231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:27.162087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:27.177954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991da12-7303-4b5b-aaba-3ab2adf49c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:27.178770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991da12-7303-4b5b-aaba-3ab2adf49c34 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.179058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991da12-7303-4b5b-aaba-3ab2adf49c34 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.179337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991da12-7303-4b5b-aaba-3ab2adf49c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.186427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d37aeb-2221-40a6-b7d0-01ec3826d6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:27.187479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d37aeb-2221-40a6-b7d0-01ec3826d6ee None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.187913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d37aeb-2221-40a6-b7d0-01ec3826d6ee None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.188939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d37aeb-2221-40a6-b7d0-01ec3826d6ee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.189875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d37aeb-2221-40a6-b7d0-01ec3826d6ee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:27.192480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d37aeb-2221-40a6-b7d0-01ec3826d6ee None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:27.192736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991da12-7303-4b5b-aaba-3ab2adf49c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:27.194251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991da12-7303-4b5b-aaba-3ab2adf49c34 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-08d37aeb-2221-40a6-b7d0-01ec3826d6ee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:27.210755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:27.211973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:35:27.213022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:27.215526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 355/707] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Jul 14 15:35:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-b991da12-7303-4b5b-aaba-3ab2adf49c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:27.215643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:27.216648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:27.217768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:27.217768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:27.217768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:27.217768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:27.217768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 353/708] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:27.222729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991da12-7303-4b5b-aaba-3ab2adf49c34 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:27.223846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d37aeb-2221-40a6-b7d0-01ec3826d6ee None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:27.426768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991da12-7303-4b5b-aaba-3ab2adf49c34 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:27.429897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d37aeb-2221-40a6-b7d0-01ec3826d6ee None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:27.485424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb8cb5-066f-41c9-a5ed-543ea8179629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:27.486053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb8cb5-066f-41c9-a5ed-543ea8179629 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.486340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb8cb5-066f-41c9-a5ed-543ea8179629 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.486619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb8cb5-066f-41c9-a5ed-543ea8179629 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.487346 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb8cb5-066f-41c9-a5ed-543ea8179629 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:27.487656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb8cb5-066f-41c9-a5ed-543ea8179629 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:27.489048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb8cb5-066f-41c9-a5ed-543ea8179629 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:27.503273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 356/709] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 14 15:35:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:27.503934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346728d-c92c-4d23-b4f7-dfc1a38a785c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:27.504622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346728d-c92c-4d23-b4f7-dfc1a38a785c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.504908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346728d-c92c-4d23-b4f7-dfc1a38a785c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.505186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346728d-c92c-4d23-b4f7-dfc1a38a785c None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.505947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346728d-c92c-4d23-b4f7-dfc1a38a785c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:27.506260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346728d-c92c-4d23-b4f7-dfc1a38a785c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:27.507650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346728d-c92c-4d23-b4f7-dfc1a38a785c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:27.509957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb8cb5-066f-41c9-a5ed-543ea8179629 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:27.525453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 354/710] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:27.531159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346728d-c92c-4d23-b4f7-dfc1a38a785c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:27.557904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:27.590341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:27.636577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb8cb5-066f-41c9-a5ed-543ea8179629 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:27.640230 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346728d-c92c-4d23-b4f7-dfc1a38a785c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:27.673959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eaba4-c224-4ce5-bd6c-1f0809f93343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:27.674850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eaba4-c224-4ce5-bd6c-1f0809f93343 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.675228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eaba4-c224-4ce5-bd6c-1f0809f93343 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.675522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eaba4-c224-4ce5-bd6c-1f0809f93343 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.675792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590e768-17a9-4ca3-9e38-ce1446be05a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:27.676228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eaba4-c224-4ce5-bd6c-1f0809f93343 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:27.676540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eaba4-c224-4ce5-bd6c-1f0809f93343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:27.676643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e768-17a9-4ca3-9e38-ce1446be05a9 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.676872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e768-17a9-4ca3-9e38-ce1446be05a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.677167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e768-17a9-4ca3-9e38-ce1446be05a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.678133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eaba4-c224-4ce5-bd6c-1f0809f93343 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:27.684722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590e768-17a9-4ca3-9e38-ce1446be05a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:27.684859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 357/711] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 14 15:35:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:27.686216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590e768-17a9-4ca3-9e38-ce1446be05a9 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:27.686796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4215c-a075-45f2-ad9f-e51df83460b6 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.687231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4215c-a075-45f2-ad9f-e51df83460b6 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.687659 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4215c-a075-45f2-ad9f-e51df83460b6 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.694618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 358/712] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:27.696190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8f8de-90b0-4799-9d09-f416e50790a6 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.696493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8f8de-90b0-4799-9d09-f416e50790a6 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.696766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8f8de-90b0-4799-9d09-f416e50790a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-1590e768-17a9-4ca3-9e38-ce1446be05a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:27.703982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:27.705047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:27.707073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:27.707073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:27.707073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:27.707073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:27.707073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:27.707073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:27.707073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:27.707073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:27.707073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 355/713] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:27.712240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590e768-17a9-4ca3-9e38-ce1446be05a9 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:27.716453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-edb8f8de-90b0-4799-9d09-f416e50790a6 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:27.717666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-edb8f8de-90b0-4799-9d09-f416e50790a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:27.796642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 359/714] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:27.802524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:27.818775 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8f8de-90b0-4799-9d09-f416e50790a6 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:27.821709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590e768-17a9-4ca3-9e38-ce1446be05a9 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:27.861409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebc105-76ab-4c51-9b13-214930bfde90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:27.862163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebc105-76ab-4c51-9b13-214930bfde90 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.862452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebc105-76ab-4c51-9b13-214930bfde90 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.862732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebc105-76ab-4c51-9b13-214930bfde90 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.863542 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebc105-76ab-4c51-9b13-214930bfde90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:27.863854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebc105-76ab-4c51-9b13-214930bfde90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:27.865312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebc105-76ab-4c51-9b13-214930bfde90 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:27.870340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:27.878707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 356/715] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:27.884791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebc105-76ab-4c51-9b13-214930bfde90 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:27.934886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8f8de-90b0-4799-9d09-f416e50790a6 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:27.974400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39196f78-8f83-4ef1-96bd-c4367fddde4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:27.975503 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39196f78-8f83-4ef1-96bd-c4367fddde4d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:27.975969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39196f78-8f83-4ef1-96bd-c4367fddde4d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:27.976397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39196f78-8f83-4ef1-96bd-c4367fddde4d None admin] PATH_INFO: `/v3/services/network` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:27.977319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39196f78-8f83-4ef1-96bd-c4367fddde4d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:27.978999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39196f78-8f83-4ef1-96bd-c4367fddde4d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-39196f78-8f83-4ef1-96bd-c4367fddde4d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:27.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:27.994099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:35:27.995051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:27.995801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebc105-76ab-4c51-9b13-214930bfde90 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:27.998364 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 360/716] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:27 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:28.004574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39196f78-8f83-4ef1-96bd-c4367fddde4d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:28.032285 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d2056-1fd0-4cc7-aaa9-ff53f19a78f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:28.032954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d2056-1fd0-4cc7-aaa9-ff53f19a78f0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:28.033241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d2056-1fd0-4cc7-aaa9-ff53f19a78f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:28.033520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d2056-1fd0-4cc7-aaa9-ff53f19a78f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:28.034392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d2056-1fd0-4cc7-aaa9-ff53f19a78f0 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=ce88c973feef4627a1b871ee9cb26511) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:28.034715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d2056-1fd0-4cc7-aaa9-ff53f19a78f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:28.036163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d2056-1fd0-4cc7-aaa9-ff53f19a78f0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:28.048494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 357/717] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:27 2022] GET /identity/v3/role_assignments?user.id=ce88c973feef4627a1b871ee9cb26511&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=abfb60dccca7470295258595a66a5d17 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:28.171821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39196f78-8f83-4ef1-96bd-c4367fddde4d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:28.175164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:28.208854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecd405-f771-476a-9c3d-92142526a6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:28.209552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd405-f771-476a-9c3d-92142526a6a8 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:28.209855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd405-f771-476a-9c3d-92142526a6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:28.210140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd405-f771-476a-9c3d-92142526a6a8 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:28.210928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecd405-f771-476a-9c3d-92142526a6a8 None admin] RBAC: Adding query filter params (name=network) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:28.211237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecd405-f771-476a-9c3d-92142526a6a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:28.212671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecd405-f771-476a-9c3d-92142526a6a8 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:28.224186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 361/718] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:35:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:28.230245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d2056-1fd0-4cc7-aaa9-ff53f19a78f0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:28.243285 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:28.372653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d2056-1fd0-4cc7-aaa9-ff53f19a78f0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:28.415296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e277c9-8fd8-4698-97cd-e84d0fe05f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:28.415972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e277c9-8fd8-4698-97cd-e84d0fe05f79 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:28.416256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e277c9-8fd8-4698-97cd-e84d0fe05f79 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:28.416533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e277c9-8fd8-4698-97cd-e84d0fe05f79 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:28.417257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e277c9-8fd8-4698-97cd-e84d0fe05f79 None admin] RBAC: Adding query filter params (type=network) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:28.417567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e277c9-8fd8-4698-97cd-e84d0fe05f79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:28.419519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e277c9-8fd8-4698-97cd-e84d0fe05f79 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:28.425843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 358/719] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:35:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:29.300033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ba06d-0e4e-4eff-9373-01a74cf2863e None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:29.300548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ba06d-0e4e-4eff-9373-01a74cf2863e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:29.300548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ba06d-0e4e-4eff-9373-01a74cf2863e None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:29.302159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 362/720] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:29.307408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f54a0e-4e42-4b76-84ce-53c162b8e785 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:29.307796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f54a0e-4e42-4b76-84ce-53c162b8e785 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:29.307964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f54a0e-4e42-4b76-84ce-53c162b8e785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:29.331276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e4f54a0e-4e42-4b76-84ce-53c162b8e785 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:29.332476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e4f54a0e-4e42-4b76-84ce-53c162b8e785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:29.417195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:29.427217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 359/721] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 15:35:29.462703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ba06d-0e4e-4eff-9373-01a74cf2863e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:29.530358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75675ea7-f90a-4305-b01e-3abc3ef647e1 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:29.530704 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75675ea7-f90a-4305-b01e-3abc3ef647e1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:29.530924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75675ea7-f90a-4305-b01e-3abc3ef647e1 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:29.533487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 360/722] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:29.537524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a34d-95e7-42b6-b8f3-8dfe1246dc05 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:29.537834 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a34d-95e7-42b6-b8f3-8dfe1246dc05 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:29.538135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a34d-95e7-42b6-b8f3-8dfe1246dc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:29.570015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ea15a34d-95e7-42b6-b8f3-8dfe1246dc05 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:29.571197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ea15a34d-95e7-42b6-b8f3-8dfe1246dc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:29.608821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ba06d-0e4e-4eff-9373-01a74cf2863e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:29.645788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3389ebc-13a1-4048-8968-02e3045eaba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:29.647092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3389ebc-13a1-4048-8968-02e3045eaba4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:29.647577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3389ebc-13a1-4048-8968-02e3045eaba4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:29.648075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3389ebc-13a1-4048-8968-02e3045eaba4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:29.659748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3389ebc-13a1-4048-8968-02e3045eaba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-b3389ebc-13a1-4048-8968-02e3045eaba4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:29.669530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:29.670547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:29.671382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:29.674128 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 363/723] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:29.679636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3389ebc-13a1-4048-8968-02e3045eaba4 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:29.711636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 361/724] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:29.730904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:29.757397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15a34d-95e7-42b6-b8f3-8dfe1246dc05 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:29.861644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3389ebc-13a1-4048-8968-02e3045eaba4 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:29.919697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15a34d-95e7-42b6-b8f3-8dfe1246dc05 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:29.927474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b7b5d-8191-476a-a1bb-b0c6af5a7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:29.928175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b7b5d-8191-476a-a1bb-b0c6af5a7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:29.928462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b7b5d-8191-476a-a1bb-b0c6af5a7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:29.928743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b7b5d-8191-476a-a1bb-b0c6af5a7bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:29.929506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b7b5d-8191-476a-a1bb-b0c6af5a7bf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:29.929835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b7b5d-8191-476a-a1bb-b0c6af5a7bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:29.931218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b7b5d-8191-476a-a1bb-b0c6af5a7bf2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:29.951013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:29.952543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 364/725] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:29.957846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b7b5d-8191-476a-a1bb-b0c6af5a7bf2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:29.962630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97578572-a4e0-4e54-b9c7-ac4a23418dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:29.963513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97578572-a4e0-4e54-b9c7-ac4a23418dc6 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:29.963797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97578572-a4e0-4e54-b9c7-ac4a23418dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:29.964071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97578572-a4e0-4e54-b9c7-ac4a23418dc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:29.982650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97578572-a4e0-4e54-b9c7-ac4a23418dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-97578572-a4e0-4e54-b9c7-ac4a23418dc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:29.990866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:29.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:29.992678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 362/726] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:30.001848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97578572-a4e0-4e54-b9c7-ac4a23418dc6 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:30.138582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97578572-a4e0-4e54-b9c7-ac4a23418dc6 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:30.149104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:30.160052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b7b5d-8191-476a-a1bb-b0c6af5a7bf2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:30.202708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a00a3-4f9f-4a4e-a392-3a396455122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:30.203505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a00a3-4f9f-4a4e-a392-3a396455122e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.203793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a00a3-4f9f-4a4e-a392-3a396455122e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.204072 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a00a3-4f9f-4a4e-a392-3a396455122e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.205208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f0c07-3d2e-4247-8997-d5b2ad923f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:30.205908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f0c07-3d2e-4247-8997-d5b2ad923f72 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.206192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f0c07-3d2e-4247-8997-d5b2ad923f72 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.206467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f0c07-3d2e-4247-8997-d5b2ad923f72 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.207219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f0c07-3d2e-4247-8997-d5b2ad923f72 None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:30.207526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f0c07-3d2e-4247-8997-d5b2ad923f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:30.208909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f0c07-3d2e-4247-8997-d5b2ad923f72 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:30.226923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 363/727] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:30.227727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a00a3-4f9f-4a4e-a392-3a396455122e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:30.229202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a00a3-4f9f-4a4e-a392-3a396455122e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:30.238798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f0c07-3d2e-4247-8997-d5b2ad923f72 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-8c5a00a3-4f9f-4a4e-a392-3a396455122e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:30.251752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:30.252701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:30.253590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:30.253590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:30.253590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:30.253590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:30.253590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 365/728] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:30.274035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a00a3-4f9f-4a4e-a392-3a396455122e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:30.302311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:30.356311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f0c07-3d2e-4247-8997-d5b2ad923f72 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:30.391549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877ce58-a347-44bb-9275-359dff886ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:30.392339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877ce58-a347-44bb-9275-359dff886ad8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.392631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877ce58-a347-44bb-9275-359dff886ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.392915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877ce58-a347-44bb-9275-359dff886ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.404929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877ce58-a347-44bb-9275-359dff886ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:30.406556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877ce58-a347-44bb-9275-359dff886ad8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-7877ce58-a347-44bb-9275-359dff886ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:30.424008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:30.424897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:30.425794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:30.425794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:30.425794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:30.425794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:30.425794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 364/729] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:30.426841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209adbb-0672-4751-8e0d-d37b71d7f5ec None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.427136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209adbb-0672-4751-8e0d-d37b71d7f5ec None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.427422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209adbb-0672-4751-8e0d-d37b71d7f5ec None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.428948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 365/730] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:30.431100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209adbb-0672-4751-8e0d-d37b71d7f5ec None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:30.432906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a00a3-4f9f-4a4e-a392-3a396455122e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:30.477275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5035238-67b0-48cb-80cc-b9a2c8cb056a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:30.478102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5035238-67b0-48cb-80cc-b9a2c8cb056a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.478465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5035238-67b0-48cb-80cc-b9a2c8cb056a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.479118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5035238-67b0-48cb-80cc-b9a2c8cb056a None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.479860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5035238-67b0-48cb-80cc-b9a2c8cb056a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:30.480485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5035238-67b0-48cb-80cc-b9a2c8cb056a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:30.482821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5035238-67b0-48cb-80cc-b9a2c8cb056a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:30.500313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 366/731] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:30.502159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1acf59-9521-4356-a353-d2c85339cb47 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.502460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1acf59-9521-4356-a353-d2c85339cb47 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.502741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1acf59-9521-4356-a353-d2c85339cb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.508053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:30.520011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7f1acf59-9521-4356-a353-d2c85339cb47 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:30.526686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7f1acf59-9521-4356-a353-d2c85339cb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:30.558619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209adbb-0672-4751-8e0d-d37b71d7f5ec None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:30.574164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:30.601947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 367/732] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:30.603148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1acf59-9521-4356-a353-d2c85339cb47 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:30.604019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1955d-43a6-49a7-b838-18e22f2d378f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:30.604817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1955d-43a6-49a7-b838-18e22f2d378f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.605204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1955d-43a6-49a7-b838-18e22f2d378f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.606144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1955d-43a6-49a7-b838-18e22f2d378f None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.607843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1955d-43a6-49a7-b838-18e22f2d378f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:30.607843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1955d-43a6-49a7-b838-18e22f2d378f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:30.610129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1955d-43a6-49a7-b838-18e22f2d378f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:30.625065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 366/733] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:30.630459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1955d-43a6-49a7-b838-18e22f2d378f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:30.728836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1acf59-9521-4356-a353-d2c85339cb47 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:30.740013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1955d-43a6-49a7-b838-18e22f2d378f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:30.767056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:30.773694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cc5ae-06d5-4cc2-81f8-0e23afc87396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:30.774432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be09e2-42f6-4df4-9b64-2c85db7e5dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:30.774664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cc5ae-06d5-4cc2-81f8-0e23afc87396 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.774990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cc5ae-06d5-4cc2-81f8-0e23afc87396 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.775237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be09e2-42f6-4df4-9b64-2c85db7e5dcc None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.775335 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cc5ae-06d5-4cc2-81f8-0e23afc87396 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.782419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be09e2-42f6-4df4-9b64-2c85db7e5dcc None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.782419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be09e2-42f6-4df4-9b64-2c85db7e5dcc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.782850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cc5ae-06d5-4cc2-81f8-0e23afc87396 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:30.783823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be09e2-42f6-4df4-9b64-2c85db7e5dcc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:30.784288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cc5ae-06d5-4cc2-81f8-0e23afc87396 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:30.785272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be09e2-42f6-4df4-9b64-2c85db7e5dcc None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-3f5cc5ae-06d5-4cc2-81f8-0e23afc87396 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:30.799282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:30.800294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:30.801300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-81be09e2-42f6-4df4-9b64-2c85db7e5dcc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:30.801887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:30.803602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:30.804556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:30.805908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 368/734] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:30.806437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cc5ae-06d5-4cc2-81f8-0e23afc87396 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:30.806880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 367/735] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:35:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:30.808201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c7843-4f9d-44fd-91c9-c76ff149594e None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.808493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c7843-4f9d-44fd-91c9-c76ff149594e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.809365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c7843-4f9d-44fd-91c9-c76ff149594e None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.813914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 368/736] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:30.813914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c7843-4f9d-44fd-91c9-c76ff149594e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:30.841053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:30.916189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cc5ae-06d5-4cc2-81f8-0e23afc87396 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:30.921965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c7843-4f9d-44fd-91c9-c76ff149594e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:30.953330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9612b-ce69-4bd9-a4f5-4dc397879268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:30.954024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9612b-ce69-4bd9-a4f5-4dc397879268 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.954310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9612b-ce69-4bd9-a4f5-4dc397879268 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.954592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9612b-ce69-4bd9-a4f5-4dc397879268 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.955332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9612b-ce69-4bd9-a4f5-4dc397879268 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:30.956710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9612b-ce69-4bd9-a4f5-4dc397879268 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:30.969711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210303e-4f57-488e-aaef-cb9e500b4fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:30.972681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210303e-4f57-488e-aaef-cb9e500b4fcf None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:30.972681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210303e-4f57-488e-aaef-cb9e500b4fcf None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:30.972681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210303e-4f57-488e-aaef-cb9e500b4fcf None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:30.972681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210303e-4f57-488e-aaef-cb9e500b4fcf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:30.972681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210303e-4f57-488e-aaef-cb9e500b4fcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:30.973690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210303e-4f57-488e-aaef-cb9e500b4fcf None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:30.992111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 369/737] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 14 15:35:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:30.992925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210303e-4f57-488e-aaef-cb9e500b4fcf None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:31.055552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 369/738] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:35:30 2022] POST /identity/v3/services => generated 251 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:31.056839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa9e5-351f-41fe-8d1b-8aeb989150ca None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.057132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa9e5-351f-41fe-8d1b-8aeb989150ca None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.057413 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa9e5-351f-41fe-8d1b-8aeb989150ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.077307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-fadaa9e5-351f-41fe-8d1b-8aeb989150ca None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:31.078593 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fadaa9e5-351f-41fe-8d1b-8aeb989150ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:31.088355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:31.110899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210303e-4f57-488e-aaef-cb9e500b4fcf None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:31.133400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:31.165950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181f70d-50ef-4803-8799-9f9a63eb9d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:31.173470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 370/739] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:31.173655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181f70d-50ef-4803-8799-9f9a63eb9d77 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.173655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181f70d-50ef-4803-8799-9f9a63eb9d77 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.173655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181f70d-50ef-4803-8799-9f9a63eb9d77 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.173870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d3eb9-9d4c-466e-8c83-3f7b8fac4324 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.173968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181f70d-50ef-4803-8799-9f9a63eb9d77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:31.174044 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d3eb9-9d4c-466e-8c83-3f7b8fac4324 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.174123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181f70d-50ef-4803-8799-9f9a63eb9d77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:31.174194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d3eb9-9d4c-466e-8c83-3f7b8fac4324 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.174264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181f70d-50ef-4803-8799-9f9a63eb9d77 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:31.174341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 371/740] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:31.174341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d3eb9-9d4c-466e-8c83-3f7b8fac4324 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:31.191587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 370/741] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 14 15:35:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:31.193185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f2c61-d382-4168-a0ff-d5cf4fc7d280 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.193306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f2c61-d382-4168-a0ff-d5cf4fc7d280 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.193494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f2c61-d382-4168-a0ff-d5cf4fc7d280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.214018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-877f2c61-d382-4168-a0ff-d5cf4fc7d280 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:31.215409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-877f2c61-d382-4168-a0ff-d5cf4fc7d280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:31.294010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 371/742] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 15:35:31.294954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f2c61-d382-4168-a0ff-d5cf4fc7d280 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:31.310750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d3eb9-9d4c-466e-8c83-3f7b8fac4324 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:31.349109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e372f3-2d41-42b4-967d-d199758f9a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:31.349770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e372f3-2d41-42b4-967d-d199758f9a67 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.350084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e372f3-2d41-42b4-967d-d199758f9a67 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.350359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e372f3-2d41-42b4-967d-d199758f9a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.360858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:31.398249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:31.403255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f2c61-d382-4168-a0ff-d5cf4fc7d280 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:31.420666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e372f3-2d41-42b4-967d-d199758f9a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:31.422403 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e372f3-2d41-42b4-967d-d199758f9a67 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:31.438753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750797c-6d40-47ba-88ea-fee3e7ac64c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:31.439554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750797c-6d40-47ba-88ea-fee3e7ac64c8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.439838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750797c-6d40-47ba-88ea-fee3e7ac64c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.440113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750797c-6d40-47ba-88ea-fee3e7ac64c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.451635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750797c-6d40-47ba-88ea-fee3e7ac64c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-4750797c-6d40-47ba-88ea-fee3e7ac64c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:31.459279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:31.460214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:31.461059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 372/743] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:31.463000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750797c-6d40-47ba-88ea-fee3e7ac64c8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:31.533911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 372/744] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:31.534816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e372f3-2d41-42b4-967d-d199758f9a67 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:31.579174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750797c-6d40-47ba-88ea-fee3e7ac64c8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:31.606810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:31.617681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d57f9-1d59-4e6b-a49d-abad3df06760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:31.618372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d57f9-1d59-4e6b-a49d-abad3df06760 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.618665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d57f9-1d59-4e6b-a49d-abad3df06760 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.618949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d57f9-1d59-4e6b-a49d-abad3df06760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.620316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:31.646233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e372f3-2d41-42b4-967d-d199758f9a67 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:31.681308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c1615-5256-45fa-aa89-8f199d0b2536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:31.682005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c1615-5256-45fa-aa89-8f199d0b2536 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.682306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c1615-5256-45fa-aa89-8f199d0b2536 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.682561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c1615-5256-45fa-aa89-8f199d0b2536 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.683280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c1615-5256-45fa-aa89-8f199d0b2536 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:31.685214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c1615-5256-45fa-aa89-8f199d0b2536 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:31.693601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d57f9-1d59-4e6b-a49d-abad3df06760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:31.695101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d57f9-1d59-4e6b-a49d-abad3df06760 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:31.761670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 373/745] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:35:31 2022] POST /identity/v3/services => generated 240 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:31.763435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c1615-5256-45fa-aa89-8f199d0b2536 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:31.816209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 373/746] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 15:35:31.817209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d57f9-1d59-4e6b-a49d-abad3df06760 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:31.846749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:31.918961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c1615-5256-45fa-aa89-8f199d0b2536 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:31.934684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d57f9-1d59-4e6b-a49d-abad3df06760 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:31.967963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b5e51-a699-492d-92b9-9e64a4c77377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:31.969084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b5e51-a699-492d-92b9-9e64a4c77377 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.969610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b5e51-a699-492d-92b9-9e64a4c77377 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.970115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b5e51-a699-492d-92b9-9e64a4c77377 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.971192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b5e51-a699-492d-92b9-9e64a4c77377 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:31.972591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b5e51-a699-492d-92b9-9e64a4c77377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:31.974194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b5e51-a699-492d-92b9-9e64a4c77377 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:31.975965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:31.976937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336f835-a739-4ac3-8284-758ebf26efc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:31.977744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336f835-a739-4ac3-8284-758ebf26efc1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:31.978067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336f835-a739-4ac3-8284-758ebf26efc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:31.978342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336f835-a739-4ac3-8284-758ebf26efc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:31.987104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 374/747] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:31.988238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b5e51-a699-492d-92b9-9e64a4c77377 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:31.993161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336f835-a739-4ac3-8284-758ebf26efc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:31.994681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336f835-a739-4ac3-8284-758ebf26efc1 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-a336f835-a739-4ac3-8284-758ebf26efc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:32.013682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:32.014901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:32.015813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:32.015813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:32.015813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:32.015813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:32.015813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 374/748] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:32.016110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336f835-a739-4ac3-8284-758ebf26efc1 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:32.104405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b5e51-a699-492d-92b9-9e64a4c77377 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:32.114114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:32.142462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf894-77e9-4b93-b39d-d68a8de7eca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:32.143267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf894-77e9-4b93-b39d-d68a8de7eca2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:32.143546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf894-77e9-4b93-b39d-d68a8de7eca2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:32.143650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336f835-a739-4ac3-8284-758ebf26efc1 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:32.143826 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf894-77e9-4b93-b39d-d68a8de7eca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:32.156609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccbf894-77e9-4b93-b39d-d68a8de7eca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:32.158708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccbf894-77e9-4b93-b39d-d68a8de7eca2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-eccbf894-77e9-4b93-b39d-d68a8de7eca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:32.182222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:32.183167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:32.184132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:32.184132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:32.184132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:32.184132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:32.184354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36154da7-bb17-4756-b964-6fad15b2b143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:32.184896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36154da7-bb17-4756-b964-6fad15b2b143 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:32.187159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36154da7-bb17-4756-b964-6fad15b2b143 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:32.187159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36154da7-bb17-4756-b964-6fad15b2b143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:32.188169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 375/749] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:32.189424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf894-77e9-4b93-b39d-d68a8de7eca2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:32.198713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36154da7-bb17-4756-b964-6fad15b2b143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:32.201315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36154da7-bb17-4756-b964-6fad15b2b143 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-36154da7-bb17-4756-b964-6fad15b2b143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:32.226644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:32.227786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:32.228702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:32.228702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:32.228702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:32.228702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:32.228702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 375/750] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:32.229863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36154da7-bb17-4756-b964-6fad15b2b143 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:32.282199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:32.319316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf894-77e9-4b93-b39d-d68a8de7eca2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:32.357656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7868a26-6c0b-41a4-a701-9a3592f86336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:32.358279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7868a26-6c0b-41a4-a701-9a3592f86336 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:32.358566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7868a26-6c0b-41a4-a701-9a3592f86336 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:32.358844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7868a26-6c0b-41a4-a701-9a3592f86336 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:32.359577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7868a26-6c0b-41a4-a701-9a3592f86336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:32.359886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7868a26-6c0b-41a4-a701-9a3592f86336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:32.361243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7868a26-6c0b-41a4-a701-9a3592f86336 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.366432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36154da7-bb17-4756-b964-6fad15b2b143 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:32.377952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 376/751] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:32.378853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7868a26-6c0b-41a4-a701-9a3592f86336 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:32.400392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:32.411504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18393d00-c0cd-4c65-b716-4fce58af574a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:32.414964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18393d00-c0cd-4c65-b716-4fce58af574a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:32.414964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18393d00-c0cd-4c65-b716-4fce58af574a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:32.414964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18393d00-c0cd-4c65-b716-4fce58af574a None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:32.414964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18393d00-c0cd-4c65-b716-4fce58af574a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:32.414964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18393d00-c0cd-4c65-b716-4fce58af574a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:32.417074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18393d00-c0cd-4c65-b716-4fce58af574a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.431457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 376/752] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:32.432590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18393d00-c0cd-4c65-b716-4fce58af574a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:32.487963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7868a26-6c0b-41a4-a701-9a3592f86336 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:32.499323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:32.525802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc7a4-a989-4576-ab39-937ed3758c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:32.526857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc7a4-a989-4576-ab39-937ed3758c13 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:32.527853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc7a4-a989-4576-ab39-937ed3758c13 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:32.528341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc7a4-a989-4576-ab39-937ed3758c13 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:32.529290 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5cc7a4-a989-4576-ab39-937ed3758c13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:32.529863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5cc7a4-a989-4576-ab39-937ed3758c13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:32.531453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5cc7a4-a989-4576-ab39-937ed3758c13 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.548433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 377/753] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:32.550087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc7a4-a989-4576-ab39-937ed3758c13 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:32.566749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18393d00-c0cd-4c65-b716-4fce58af574a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:32.614090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b847756-350b-47cb-be90-54a4b7149116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:32.615071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b847756-350b-47cb-be90-54a4b7149116 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:32.615359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b847756-350b-47cb-be90-54a4b7149116 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:32.615636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b847756-350b-47cb-be90-54a4b7149116 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:32.625922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b847756-350b-47cb-be90-54a4b7149116 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:32.627656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b847756-350b-47cb-be90-54a4b7149116 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-0b847756-350b-47cb-be90-54a4b7149116 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:32.651924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:32.653400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:32.654686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:32.654686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:32.654686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:32.654686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:32.654686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:32.654686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:32.654686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:32.654686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:32.657975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 377/754] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:32.659052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b847756-350b-47cb-be90-54a4b7149116 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:32.668776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc7a4-a989-4576-ab39-937ed3758c13 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:32.705478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a4632-8af1-4a0d-8066-99b6be8db118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:32.706330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4632-8af1-4a0d-8066-99b6be8db118 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:32.706617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4632-8af1-4a0d-8066-99b6be8db118 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:32.706897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4632-8af1-4a0d-8066-99b6be8db118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:32.714061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a4632-8af1-4a0d-8066-99b6be8db118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:32.716111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a4632-8af1-4a0d-8066-99b6be8db118 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.726526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-2d2a4632-8af1-4a0d-8066-99b6be8db118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:32.731253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:32.732145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:32.733052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:32.733052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:32.733052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:32.733052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:32.733052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:32.733052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:32.733052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:32.733052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:32.733052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 378/755] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:32.733541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a4632-8af1-4a0d-8066-99b6be8db118 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:32.765435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:32.799194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b847756-350b-47cb-be90-54a4b7149116 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:32.839152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435c438-29c6-4c3f-89f6-95d52c09f9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:32.840257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435c438-29c6-4c3f-89f6-95d52c09f9e7 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:32.840763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435c438-29c6-4c3f-89f6-95d52c09f9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:32.841149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435c438-29c6-4c3f-89f6-95d52c09f9e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:32.848887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435c438-29c6-4c3f-89f6-95d52c09f9e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:32.850758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435c438-29c6-4c3f-89f6-95d52c09f9e7 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.854568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a4632-8af1-4a0d-8066-99b6be8db118 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-f435c438-29c6-4c3f-89f6-95d52c09f9e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:32.866232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:32.867368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:32.868578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:32.874819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 378/756] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:32.875746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435c438-29c6-4c3f-89f6-95d52c09f9e7 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:32.900092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9ef88-073c-4fe3-a912-a0eca00f7a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:32.901281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9ef88-073c-4fe3-a912-a0eca00f7a83 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:32.901721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9ef88-073c-4fe3-a912-a0eca00f7a83 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:32.902484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9ef88-073c-4fe3-a912-a0eca00f7a83 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:32.903884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9ef88-073c-4fe3-a912-a0eca00f7a83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:32.904645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9ef88-073c-4fe3-a912-a0eca00f7a83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:32.906957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9ef88-073c-4fe3-a912-a0eca00f7a83 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:32.928433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 379/757] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:32.928433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9ef88-073c-4fe3-a912-a0eca00f7a83 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:33.000049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435c438-29c6-4c3f-89f6-95d52c09f9e7 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:33.010549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:33.025083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:33.038817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9ef88-073c-4fe3-a912-a0eca00f7a83 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:33.039484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c80bc-727c-404a-b7ba-34df4796bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:33.040332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c80bc-727c-404a-b7ba-34df4796bf8c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.040772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c80bc-727c-404a-b7ba-34df4796bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.041264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c80bc-727c-404a-b7ba-34df4796bf8c None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.042314 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c80bc-727c-404a-b7ba-34df4796bf8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:33.042842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c80bc-727c-404a-b7ba-34df4796bf8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:33.044600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c80bc-727c-404a-b7ba-34df4796bf8c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:33.059205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 379/758] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:33.060365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c80bc-727c-404a-b7ba-34df4796bf8c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:33.077634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c868b0-add5-404e-a5cb-abde09dbd058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:33.078383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c868b0-add5-404e-a5cb-abde09dbd058 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.078668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c868b0-add5-404e-a5cb-abde09dbd058 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.078947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c868b0-add5-404e-a5cb-abde09dbd058 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.079806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c868b0-add5-404e-a5cb-abde09dbd058 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:33.080118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c868b0-add5-404e-a5cb-abde09dbd058 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:33.081719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c868b0-add5-404e-a5cb-abde09dbd058 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:33.104359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 380/759] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 14 15:35:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:33.105263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c868b0-add5-404e-a5cb-abde09dbd058 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:33.171016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c80bc-727c-404a-b7ba-34df4796bf8c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:33.209799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82faf87-1954-4044-8c1a-74514cd77495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:33.210869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82faf87-1954-4044-8c1a-74514cd77495 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.211367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82faf87-1954-4044-8c1a-74514cd77495 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.211875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82faf87-1954-4044-8c1a-74514cd77495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.211992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c868b0-add5-404e-a5cb-abde09dbd058 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:33.213017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82faf87-1954-4044-8c1a-74514cd77495 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=95f36b4517ee4b64bc43e000c854df30) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:33.213599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82faf87-1954-4044-8c1a-74514cd77495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:33.215435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82faf87-1954-4044-8c1a-74514cd77495 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:33.221749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 380/760] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:33 2022] GET /identity/v3/role_assignments?user.id=95f36b4517ee4b64bc43e000c854df30&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=abfb60dccca7470295258595a66a5d17 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:33.222917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82faf87-1954-4044-8c1a-74514cd77495 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:33.253461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bcf9-2af7-4eba-aa34-2e3c5061eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:33.254273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bcf9-2af7-4eba-aa34-2e3c5061eca8 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.254623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bcf9-2af7-4eba-aa34-2e3c5061eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.254974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bcf9-2af7-4eba-aa34-2e3c5061eca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.255892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bcf9-2af7-4eba-aa34-2e3c5061eca8 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=613c0627a67041cd858b61ecc3d08421) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:33.256296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bcf9-2af7-4eba-aa34-2e3c5061eca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:33.257872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bcf9-2af7-4eba-aa34-2e3c5061eca8 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:33.264708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 381/761] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:33 2022] GET /identity/v3/role_assignments?user.id=613c0627a67041cd858b61ecc3d08421&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:33.266230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4a32a-4e64-4aa6-a88f-ffcf7ec75581 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.266593 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4a32a-4e64-4aa6-a88f-ffcf7ec75581 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.266945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4a32a-4e64-4aa6-a88f-ffcf7ec75581 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.267592 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:33.268747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 382/762] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:33.281866 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36592-5eea-45f6-adc1-c66be46fe3ea None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.281866 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36592-5eea-45f6-adc1-c66be46fe3ea None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.281866 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36592-5eea-45f6-adc1-c66be46fe3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.307011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d5d36592-5eea-45f6-adc1-c66be46fe3ea None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:33.309773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d5d36592-5eea-45f6-adc1-c66be46fe3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:33.320959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:33.418898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82faf87-1954-4044-8c1a-74514cd77495 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:33.453426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 383/763] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:35:33.466280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe3a47-b0ee-49da-a4da-c59a55de82a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:33.467088 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe3a47-b0ee-49da-a4da-c59a55de82a5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.467377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe3a47-b0ee-49da-a4da-c59a55de82a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.467653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe3a47-b0ee-49da-a4da-c59a55de82a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.478279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36592-5eea-45f6-adc1-c66be46fe3ea None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:33.479189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe3a47-b0ee-49da-a4da-c59a55de82a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:33.480914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe3a47-b0ee-49da-a4da-c59a55de82a5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-50fe3a47-b0ee-49da-a4da-c59a55de82a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:33.511794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:33.512778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:33.513682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:33.513682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:33.513682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:33.513682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:33.525464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 381/764] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:33.526434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe3a47-b0ee-49da-a4da-c59a55de82a5 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:33.684512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36592-5eea-45f6-adc1-c66be46fe3ea None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:33.685035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:33.705834 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe3a47-b0ee-49da-a4da-c59a55de82a5 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:33.728436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6c3a4-0cdb-42ee-8441-570557c75057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:33.729613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6c3a4-0cdb-42ee-8441-570557c75057 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.730143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6c3a4-0cdb-42ee-8441-570557c75057 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.731180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6c3a4-0cdb-42ee-8441-570557c75057 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.732114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6c3a4-0cdb-42ee-8441-570557c75057 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:33.733741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6c3a4-0cdb-42ee-8441-570557c75057 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-9fb6c3a4-0cdb-42ee-8441-570557c75057 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:33.751572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:33.752568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:35:33.753432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:33.763287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 384/765] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Jul 14 15:35:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:33.768253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6c3a4-0cdb-42ee-8441-570557c75057 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:33.772140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8092b-e744-446b-9bfe-c97dfe643c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:33.773141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8092b-e744-446b-9bfe-c97dfe643c9a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.773640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8092b-e744-446b-9bfe-c97dfe643c9a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.774158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8092b-e744-446b-9bfe-c97dfe643c9a None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.775295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8092b-e744-446b-9bfe-c97dfe643c9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:33.776644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8092b-e744-446b-9bfe-c97dfe643c9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:33.777498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:33.780459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8092b-e744-446b-9bfe-c97dfe643c9a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:33.800203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 382/766] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:33.806117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8092b-e744-446b-9bfe-c97dfe643c9a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:33.914735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6c3a4-0cdb-42ee-8441-570557c75057 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:33.953118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f87f57-8097-4e08-aa95-6de966497d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:33.953797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f87f57-8097-4e08-aa95-6de966497d8c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:33.954113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f87f57-8097-4e08-aa95-6de966497d8c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:33.954392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f87f57-8097-4e08-aa95-6de966497d8c None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:33.955714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f87f57-8097-4e08-aa95-6de966497d8c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:33.956191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f87f57-8097-4e08-aa95-6de966497d8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:33.957675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f87f57-8097-4e08-aa95-6de966497d8c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:33.962769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 385/767] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 14 15:35:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:33.970775 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f87f57-8097-4e08-aa95-6de966497d8c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:34.027850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8092b-e744-446b-9bfe-c97dfe643c9a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:34.070000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ac6dd-a858-4f1f-8ed9-f82476d3f0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:34.075799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ac6dd-a858-4f1f-8ed9-f82476d3f0da None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.075799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ac6dd-a858-4f1f-8ed9-f82476d3f0da None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.075799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ac6dd-a858-4f1f-8ed9-f82476d3f0da None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.079751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:34.081517 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ac6dd-a858-4f1f-8ed9-f82476d3f0da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:34.083498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ac6dd-a858-4f1f-8ed9-f82476d3f0da None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-7d4ac6dd-a858-4f1f-8ed9-f82476d3f0da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:34.107468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:34.108622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:34.109722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:34.109722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:34.109722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:34.109722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:34.109722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:34.109722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:34.109722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:34.109722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:34.111232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 383/768] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:34.112838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0087e-c58f-4a90-8b90-19e2381f3777 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.113280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0087e-c58f-4a90-8b90-19e2381f3777 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.113628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0087e-c58f-4a90-8b90-19e2381f3777 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.119536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 384/769] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:34.119536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0087e-c58f-4a90-8b90-19e2381f3777 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:34.130526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f87f57-8097-4e08-aa95-6de966497d8c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:34.138202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:34.175830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff20f3-e0f7-4d43-8a28-1ed591d37452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:34.177121 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff20f3-e0f7-4d43-8a28-1ed591d37452 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.177859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff20f3-e0f7-4d43-8a28-1ed591d37452 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.178322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff20f3-e0f7-4d43-8a28-1ed591d37452 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.179374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff20f3-e0f7-4d43-8a28-1ed591d37452 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:34.180142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff20f3-e0f7-4d43-8a28-1ed591d37452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:34.182367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff20f3-e0f7-4d43-8a28-1ed591d37452 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:34.189661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 386/770] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 14 15:35:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:34.191500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069f2ac-d68a-465c-9dcc-cfac6b69dc8c None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.191955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069f2ac-d68a-465c-9dcc-cfac6b69dc8c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.192397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069f2ac-d68a-465c-9dcc-cfac6b69dc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.214362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d069f2ac-d68a-465c-9dcc-cfac6b69dc8c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:34.216103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d069f2ac-d68a-465c-9dcc-cfac6b69dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:34.240559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0087e-c58f-4a90-8b90-19e2381f3777 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:34.285693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35accac0-8a57-4e5b-a5d6-745f173d0978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:34.286506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35accac0-8a57-4e5b-a5d6-745f173d0978 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.286956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35accac0-8a57-4e5b-a5d6-745f173d0978 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.287305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35accac0-8a57-4e5b-a5d6-745f173d0978 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.288220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35accac0-8a57-4e5b-a5d6-745f173d0978 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:34.288596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35accac0-8a57-4e5b-a5d6-745f173d0978 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:34.289180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 387/771] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:34.290220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069f2ac-d68a-465c-9dcc-cfac6b69dc8c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:34.294760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35accac0-8a57-4e5b-a5d6-745f173d0978 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:34.319588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 385/772] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:34.325454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35accac0-8a57-4e5b-a5d6-745f173d0978 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:34.368895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:34.393886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:34.421697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069f2ac-d68a-465c-9dcc-cfac6b69dc8c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:34.436838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35accac0-8a57-4e5b-a5d6-745f173d0978 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:34.470799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ad609-3e55-447e-b6b8-8868c1c44cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:34.471962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ad609-3e55-447e-b6b8-8868c1c44cdd None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.472512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ad609-3e55-447e-b6b8-8868c1c44cdd None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.473007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ad609-3e55-447e-b6b8-8868c1c44cdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.474159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ad609-3e55-447e-b6b8-8868c1c44cdd None admin] RBAC: Adding query filter params (interface=public, service_id=5e59c80e1b834bc49f5293b62338316d, region_id=RegionOne) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:34.474269 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee69798-1d3c-45d8-8d8f-97d9cfa2156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:34.474795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ad609-3e55-447e-b6b8-8868c1c44cdd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:34.475089 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee69798-1d3c-45d8-8d8f-97d9cfa2156c None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.475378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee69798-1d3c-45d8-8d8f-97d9cfa2156c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.475660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee69798-1d3c-45d8-8d8f-97d9cfa2156c None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/ce88c973feef4627a1b871ee9cb26511/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.477264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ad609-3e55-447e-b6b8-8868c1c44cdd None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:34.477729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee69798-1d3c-45d8-8d8f-97d9cfa2156c None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=ce88c973feef4627a1b871ee9cb26511, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:34.479244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee69798-1d3c-45d8-8d8f-97d9cfa2156c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:34.489282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 388/773] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 14 15:35:34 2022] GET /identity/v3/endpoints?service_id=5e59c80e1b834bc49f5293b62338316d&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:34.490710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ad609-3e55-447e-b6b8-8868c1c44cdd None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:34.547049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 386/774] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:35:34 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/ce88c973feef4627a1b871ee9cb26511/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:35:34.548321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee69798-1d3c-45d8-8d8f-97d9cfa2156c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:34.675950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee69798-1d3c-45d8-8d8f-97d9cfa2156c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:34.676155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ad609-3e55-447e-b6b8-8868c1c44cdd None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:34.706759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:34.719871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec6a87-1473-4552-ab69-94e36919a84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:34.721050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec6a87-1473-4552-ab69-94e36919a84e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.721555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec6a87-1473-4552-ab69-94e36919a84e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.722293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec6a87-1473-4552-ab69-94e36919a84e None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.723657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec6a87-1473-4552-ab69-94e36919a84e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:34.725292 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec6a87-1473-4552-ab69-94e36919a84e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:34.732592 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 387/775] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 14 15:35:34 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:34.742028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:34.757330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd3e9-32f9-4663-a960-8bcae1859ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:34.758636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd3e9-32f9-4663-a960-8bcae1859ed2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.759198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd3e9-32f9-4663-a960-8bcae1859ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.759680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd3e9-32f9-4663-a960-8bcae1859ed2 None admin] PATH_INFO: `/v3/services/network` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.760605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edd3e9-32f9-4663-a960-8bcae1859ed2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:34.762376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edd3e9-32f9-4663-a960-8bcae1859ed2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-f6edd3e9-32f9-4663-a960-8bcae1859ed2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:34.779787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:34.780851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:35:34.781711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:34.785142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 389/776] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:34 2022] GET /identity/v3/services/network => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:34.790519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec6a87-1473-4552-ab69-94e36919a84e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:34.906947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec6a87-1473-4552-ab69-94e36919a84e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:34.944759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dce67-85e0-4bf4-bbdc-e205491d186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:34.945846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dce67-85e0-4bf4-bbdc-e205491d186f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:34.946360 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dce67-85e0-4bf4-bbdc-e205491d186f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:34.946833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dce67-85e0-4bf4-bbdc-e205491d186f None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:34.947757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dce67-85e0-4bf4-bbdc-e205491d186f None admin] RBAC: Adding query filter params (name=network) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:34.948276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dce67-85e0-4bf4-bbdc-e205491d186f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:34.950031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dce67-85e0-4bf4-bbdc-e205491d186f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:34.955691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 388/777] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:35:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:34.961013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd3e9-32f9-4663-a960-8bcae1859ed2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:35.069781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd3e9-32f9-4663-a960-8bcae1859ed2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:35.105387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51fb6d-4267-4b58-9b2a-f4831ebc9456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:35.106679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51fb6d-4267-4b58-9b2a-f4831ebc9456 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:35.106960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51fb6d-4267-4b58-9b2a-f4831ebc9456 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:35.107238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51fb6d-4267-4b58-9b2a-f4831ebc9456 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:35.108008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51fb6d-4267-4b58-9b2a-f4831ebc9456 None admin] RBAC: Adding query filter params (type=network) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:35.108320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51fb6d-4267-4b58-9b2a-f4831ebc9456 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:35.109693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51fb6d-4267-4b58-9b2a-f4831ebc9456 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:35.115229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 390/778] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:35:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:35.122905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dce67-85e0-4bf4-bbdc-e205491d186f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:35.186203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:35.228232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dce67-85e0-4bf4-bbdc-e205491d186f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:35.264686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacaabc-16c2-4003-8327-8ddfdf0af12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:35.265359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacaabc-16c2-4003-8327-8ddfdf0af12b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:35.265645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacaabc-16c2-4003-8327-8ddfdf0af12b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:35.265958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacaabc-16c2-4003-8327-8ddfdf0af12b None admin] PATH_INFO: `/v3/endpoints` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:35.266759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacaabc-16c2-4003-8327-8ddfdf0af12b None admin] RBAC: Adding query filter params (interface=public, service_id=96fde356f0a3479cbf031bec9107a60e, region_id=RegionOne) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:35.267073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacaabc-16c2-4003-8327-8ddfdf0af12b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:35.268440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacaabc-16c2-4003-8327-8ddfdf0af12b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:35.276713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 389/779] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 14 15:35:35 2022] GET /identity/v3/endpoints?service_id=96fde356f0a3479cbf031bec9107a60e&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:35.286931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51fb6d-4267-4b58-9b2a-f4831ebc9456 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:35.366306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:35.411823 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51fb6d-4267-4b58-9b2a-f4831ebc9456 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:35.446840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45773c8c-877f-4a07-95b8-7274c252f3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:35.447575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45773c8c-877f-4a07-95b8-7274c252f3af None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:35.447858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45773c8c-877f-4a07-95b8-7274c252f3af None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:35.448139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45773c8c-877f-4a07-95b8-7274c252f3af None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:35.448866 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45773c8c-877f-4a07-95b8-7274c252f3af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:35.450407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45773c8c-877f-4a07-95b8-7274c252f3af None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:35.456245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 391/780] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 14 15:35:35 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:35.459245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a163a3-2fbc-4694-b302-82fe7a77f9df None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:35.459727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a163a3-2fbc-4694-b302-82fe7a77f9df None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:35.459808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a163a3-2fbc-4694-b302-82fe7a77f9df None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:35.461418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 390/781] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:35.467979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103132c-19d1-4937-b454-52d120061558 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:35.468278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103132c-19d1-4937-b454-52d120061558 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:35.468550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103132c-19d1-4937-b454-52d120061558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:35.486899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d103132c-19d1-4937-b454-52d120061558 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:35.488138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d103132c-19d1-4937-b454-52d120061558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:35.566006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 392/782] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:35.584760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a163a3-2fbc-4694-b302-82fe7a77f9df None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:35.642670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de2ccd-2ef4-4746-9c5d-c75e5a7896c9 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:35.642973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de2ccd-2ef4-4746-9c5d-c75e5a7896c9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:35.643141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de2ccd-2ef4-4746-9c5d-c75e5a7896c9 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:35.644951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 393/783] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:35.660300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e31d84-b62a-47ce-9a04-e4bfc93d78c3 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:35.660614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e31d84-b62a-47ce-9a04-e4bfc93d78c3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:35.660905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e31d84-b62a-47ce-9a04-e4bfc93d78c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:35.707694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a163a3-2fbc-4694-b302-82fe7a77f9df None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:35.715369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-79e31d84-b62a-47ce-9a04-e4bfc93d78c3 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:35.716616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-79e31d84-b62a-47ce-9a04-e4bfc93d78c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:35.731489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:35.749778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adae74-f56b-4cb3-98c4-6b0c7705edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:35.750036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adae74-f56b-4cb3-98c4-6b0c7705edb7 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:35.750335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adae74-f56b-4cb3-98c4-6b0c7705edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:35.750604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adae74-f56b-4cb3-98c4-6b0c7705edb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:35.763846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5adae74-f56b-4cb3-98c4-6b0c7705edb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:35.769617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-c5adae74-f56b-4cb3-98c4-6b0c7705edb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:35.773140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:35.774349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:35.775454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:35.778324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 391/784] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:35.782079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adae74-f56b-4cb3-98c4-6b0c7705edb7 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:35.803872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 394/785] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:35.838798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e31d84-b62a-47ce-9a04-e4bfc93d78c3 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:35.900161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adae74-f56b-4cb3-98c4-6b0c7705edb7 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:35.937025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6f6fb-472d-4afd-8a07-60adc7cb0bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:35.937705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6f6fb-472d-4afd-8a07-60adc7cb0bde None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:35.938026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6f6fb-472d-4afd-8a07-60adc7cb0bde None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:35.938302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6f6fb-472d-4afd-8a07-60adc7cb0bde None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:35.939375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6f6fb-472d-4afd-8a07-60adc7cb0bde None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:35.939690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6f6fb-472d-4afd-8a07-60adc7cb0bde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:35.941487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6f6fb-472d-4afd-8a07-60adc7cb0bde None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:35.955051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 392/786] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:35.957461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6f6fb-472d-4afd-8a07-60adc7cb0bde None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:36.006187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:36.070788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6f6fb-472d-4afd-8a07-60adc7cb0bde None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:36.110156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad814f5f-28dc-45e3-9b1b-371a52219c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:36.111640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad814f5f-28dc-45e3-9b1b-371a52219c7f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:36.112011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad814f5f-28dc-45e3-9b1b-371a52219c7f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:36.112594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad814f5f-28dc-45e3-9b1b-371a52219c7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:36.120580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e31d84-b62a-47ce-9a04-e4bfc93d78c3 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:36.126380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad814f5f-28dc-45e3-9b1b-371a52219c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:36.127864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad814f5f-28dc-45e3-9b1b-371a52219c7f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-ad814f5f-28dc-45e3-9b1b-371a52219c7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:36.147410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:36.148398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:36.149307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:36.149307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:36.149307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:36.149307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:36.153371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 393/787] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:36.154545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad814f5f-28dc-45e3-9b1b-371a52219c7f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:36.162067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec4d2c-44e7-459c-9eb5-2a4925ba6246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:36.163249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec4d2c-44e7-459c-9eb5-2a4925ba6246 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:36.163743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec4d2c-44e7-459c-9eb5-2a4925ba6246 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:36.164213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec4d2c-44e7-459c-9eb5-2a4925ba6246 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:36.176091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec4d2c-44e7-459c-9eb5-2a4925ba6246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-31ec4d2c-44e7-459c-9eb5-2a4925ba6246 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:36.185644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:36.186711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:36.187587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:36.191090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 395/788] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:36.196523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec4d2c-44e7-459c-9eb5-2a4925ba6246 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:36.274164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:36.316145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:36.337032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad814f5f-28dc-45e3-9b1b-371a52219c7f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:36.380169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec4d2c-44e7-459c-9eb5-2a4925ba6246 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:36.403147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63758040-5e64-43da-814a-3e1deb160f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:36.403876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63758040-5e64-43da-814a-3e1deb160f23 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:36.404102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63758040-5e64-43da-814a-3e1deb160f23 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:36.404380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63758040-5e64-43da-814a-3e1deb160f23 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:36.405106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63758040-5e64-43da-814a-3e1deb160f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:36.405412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63758040-5e64-43da-814a-3e1deb160f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:36.406817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63758040-5e64-43da-814a-3e1deb160f23 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:36.426616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 394/789] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:36.432138 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63758040-5e64-43da-814a-3e1deb160f23 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:36.444790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982b18e-9980-4ee3-9179-22a0ffa755c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:36.445469 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982b18e-9980-4ee3-9179-22a0ffa755c5 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:36.445751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982b18e-9980-4ee3-9179-22a0ffa755c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:36.446060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982b18e-9980-4ee3-9179-22a0ffa755c5 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:36.446812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982b18e-9980-4ee3-9179-22a0ffa755c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:36.447119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982b18e-9980-4ee3-9179-22a0ffa755c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:36.448492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982b18e-9980-4ee3-9179-22a0ffa755c5 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:36.473333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 396/790] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:36.478753 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982b18e-9980-4ee3-9179-22a0ffa755c5 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:36.558369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63758040-5e64-43da-814a-3e1deb160f23 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:36.625475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36143313-c4ae-43fc-802a-bb94adad8c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:36.628309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36143313-c4ae-43fc-802a-bb94adad8c07 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:36.628782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36143313-c4ae-43fc-802a-bb94adad8c07 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:36.629239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36143313-c4ae-43fc-802a-bb94adad8c07 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:36.645309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36143313-c4ae-43fc-802a-bb94adad8c07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:36.648862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36143313-c4ae-43fc-802a-bb94adad8c07 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:36.671949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:36.675531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982b18e-9980-4ee3-9179-22a0ffa755c5 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-36143313-c4ae-43fc-802a-bb94adad8c07 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:36.682180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:36.691645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 395/791] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:35:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:36.692597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36143313-c4ae-43fc-802a-bb94adad8c07 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:36.726223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:36.750947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfb602-f0e7-410f-ad6a-1c35ce980a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:36.752060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfb602-f0e7-410f-ad6a-1c35ce980a04 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:36.752344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfb602-f0e7-410f-ad6a-1c35ce980a04 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:36.752620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfb602-f0e7-410f-ad6a-1c35ce980a04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:36.774213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfb602-f0e7-410f-ad6a-1c35ce980a04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:36.777135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfb602-f0e7-410f-ad6a-1c35ce980a04 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-d2bfb602-f0e7-410f-ad6a-1c35ce980a04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:36.812188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:36.813150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:36.814078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:36.814078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:36.814078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:36.814078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:36.818892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 397/792] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:36.823699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfb602-f0e7-410f-ad6a-1c35ce980a04 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:36.862199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36143313-c4ae-43fc-802a-bb94adad8c07 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:36.934832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c49fc5-8681-4c03-853b-be0ba6d73b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:36.935663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49fc5-8681-4c03-853b-be0ba6d73b44 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:36.935891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49fc5-8681-4c03-853b-be0ba6d73b44 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:36.936171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49fc5-8681-4c03-853b-be0ba6d73b44 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:36.937019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c49fc5-8681-4c03-853b-be0ba6d73b44 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:36.937331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c49fc5-8681-4c03-853b-be0ba6d73b44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:36.938972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c49fc5-8681-4c03-853b-be0ba6d73b44 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:36.942107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfb602-f0e7-410f-ad6a-1c35ce980a04 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:36.976452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 396/793] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 14 15:35:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:36.981478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c49fc5-8681-4c03-853b-be0ba6d73b44 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:36.983693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f9131-93e6-408c-a9d4-9e53199e6e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:36.984625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f9131-93e6-408c-a9d4-9e53199e6e1d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:36.985110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f9131-93e6-408c-a9d4-9e53199e6e1d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:36.985590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f9131-93e6-408c-a9d4-9e53199e6e1d None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:36.986720 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f9131-93e6-408c-a9d4-9e53199e6e1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:36.987250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f9131-93e6-408c-a9d4-9e53199e6e1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:36.988882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f9131-93e6-408c-a9d4-9e53199e6e1d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:37.003393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 398/794] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:37.011276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f9131-93e6-408c-a9d4-9e53199e6e1d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:37.036238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:37.120200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:37.179714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c49fc5-8681-4c03-853b-be0ba6d73b44 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:37.221244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f9131-93e6-408c-a9d4-9e53199e6e1d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:37.236324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c73a3-0789-498b-a8c2-f253395d4e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:37.237133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c73a3-0789-498b-a8c2-f253395d4e91 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.237418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c73a3-0789-498b-a8c2-f253395d4e91 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.237697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c73a3-0789-498b-a8c2-f253395d4e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.255165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c73a3-0789-498b-a8c2-f253395d4e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:37.256654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c73a3-0789-498b-a8c2-f253395d4e91 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-301c73a3-0789-498b-a8c2-f253395d4e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:37.278419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:37.279404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:37.280734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:37.280734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:37.280734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:37.280734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:37.280734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 397/795] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:37.285331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c73a3-0789-498b-a8c2-f253395d4e91 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:37.290586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe746f4-ceb4-44c1-a69c-3ac6ebb17c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:37.291687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe746f4-ceb4-44c1-a69c-3ac6ebb17c7b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.292185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe746f4-ceb4-44c1-a69c-3ac6ebb17c7b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.292661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe746f4-ceb4-44c1-a69c-3ac6ebb17c7b None admin] PATH_INFO: `/v3/users/swift` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.306983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe746f4-ceb4-44c1-a69c-3ac6ebb17c7b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:37.309315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe746f4-ceb4-44c1-a69c-3ac6ebb17c7b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-6fe746f4-ceb4-44c1-a69c-3ac6ebb17c7b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:37.329029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:37.330045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:37.331417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 399/796] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:37.332012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beeba5-9a34-4b4c-ae88-1f093351293b None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.332409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beeba5-9a34-4b4c-ae88-1f093351293b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.332409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6beeba5-9a34-4b4c-ae88-1f093351293b None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.333956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 400/797] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:37.337628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6beeba5-9a34-4b4c-ae88-1f093351293b None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:37.421717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c73a3-0789-498b-a8c2-f253395d4e91 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:37.446398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:37.447827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6beeba5-9a34-4b4c-ae88-1f093351293b None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:37.448808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:37.464445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8869ad7-6ca2-44ef-89f8-4172c6728230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:37.465847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8869ad7-6ca2-44ef-89f8-4172c6728230 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.477066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8869ad7-6ca2-44ef-89f8-4172c6728230 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.477066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8869ad7-6ca2-44ef-89f8-4172c6728230 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.477066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8869ad7-6ca2-44ef-89f8-4172c6728230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:37.477066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8869ad7-6ca2-44ef-89f8-4172c6728230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:37.477066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8869ad7-6ca2-44ef-89f8-4172c6728230 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:37.481157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 398/798] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:37.482606 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e183980-de4a-4551-9c5b-f4a92d2d3179 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.482894 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e183980-de4a-4551-9c5b-f4a92d2d3179 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.483171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e183980-de4a-4551-9c5b-f4a92d2d3179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.488299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad48b4-2273-489e-8388-bc51c19d09ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:37.489201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad48b4-2273-489e-8388-bc51c19d09ac None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.489672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad48b4-2273-489e-8388-bc51c19d09ac None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.489985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad48b4-2273-489e-8388-bc51c19d09ac None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.490834 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad48b4-2273-489e-8388-bc51c19d09ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:37.491145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad48b4-2273-489e-8388-bc51c19d09ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:37.492567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad48b4-2273-489e-8388-bc51c19d09ac None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:37.500304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5e183980-de4a-4551-9c5b-f4a92d2d3179 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:37.501505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5e183980-de4a-4551-9c5b-f4a92d2d3179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:37.515668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 401/799] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 14 15:35:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:37.515668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdce94-a7fe-4a56-81a9-51301252f347 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.515668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdce94-a7fe-4a56-81a9-51301252f347 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.515668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdce94-a7fe-4a56-81a9-51301252f347 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.515668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 402/800] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:37.515668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdce94-a7fe-4a56-81a9-51301252f347 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:37.579943 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 399/801] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:37.581045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e183980-de4a-4551-9c5b-f4a92d2d3179 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:37.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdce94-a7fe-4a56-81a9-51301252f347 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:37.662860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:37.668680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2a8d1-6c70-4a32-92a4-92d20fd34134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:37.669525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2a8d1-6c70-4a32-92a4-92d20fd34134 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.669766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2a8d1-6c70-4a32-92a4-92d20fd34134 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.670069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2a8d1-6c70-4a32-92a4-92d20fd34134 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.677096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2a8d1-6c70-4a32-92a4-92d20fd34134 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:37.678571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2a8d1-6c70-4a32-92a4-92d20fd34134 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:37.689963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e183980-de4a-4551-9c5b-f4a92d2d3179 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-24d2a8d1-6c70-4a32-92a4-92d20fd34134 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:37.694061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:37.694937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:37.695830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:37.695830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:37.695830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:37.695830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:37.695830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:37.695830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:37.695830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:37.695830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:37.700454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 403/802] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:37.701686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bd59b-4356-459e-9019-03d4f1dde5a6 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.702007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bd59b-4356-459e-9019-03d4f1dde5a6 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.702295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bd59b-4356-459e-9019-03d4f1dde5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.702834 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:37.720412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-2e3bd59b-4356-459e-9019-03d4f1dde5a6 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:37.721744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2e3bd59b-4356-459e-9019-03d4f1dde5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:37.726729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700950d-1ec7-461f-97b3-248cab7fb765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:37.727583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700950d-1ec7-461f-97b3-248cab7fb765 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.727819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700950d-1ec7-461f-97b3-248cab7fb765 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.728098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700950d-1ec7-461f-97b3-248cab7fb765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.738900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700950d-1ec7-461f-97b3-248cab7fb765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:37.740362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700950d-1ec7-461f-97b3-248cab7fb765 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-c700950d-1ec7-461f-97b3-248cab7fb765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:37.758985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:37.759913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:37.760813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:37.760813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:37.760813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:37.760813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:37.760813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 400/803] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:37.761088 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700950d-1ec7-461f-97b3-248cab7fb765 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:37.800305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 404/804] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:37.800849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bd59b-4356-459e-9019-03d4f1dde5a6 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:37.874908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700950d-1ec7-461f-97b3-248cab7fb765 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:37.898844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bd59b-4356-459e-9019-03d4f1dde5a6 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:37.908746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1178a-7e05-4bf2-a5db-3585e752d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:37.909593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1178a-7e05-4bf2-a5db-3585e752d7f1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.909905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1178a-7e05-4bf2-a5db-3585e752d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.910190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1178a-7e05-4bf2-a5db-3585e752d7f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.918310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:37.920618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1178a-7e05-4bf2-a5db-3585e752d7f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-7ec1178a-7e05-4bf2-a5db-3585e752d7f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:37.928262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:37.933378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 401/805] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:37.934783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1178a-7e05-4bf2-a5db-3585e752d7f1 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:37.935552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e277e8-c1f6-4e2a-acb0-64611b933ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:37.935552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e277e8-c1f6-4e2a-acb0-64611b933ac9 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:37.935552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e277e8-c1f6-4e2a-acb0-64611b933ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:37.935552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e277e8-c1f6-4e2a-acb0-64611b933ac9 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:37.935903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e277e8-c1f6-4e2a-acb0-64611b933ac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:37.935903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e277e8-c1f6-4e2a-acb0-64611b933ac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:37.936049 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e277e8-c1f6-4e2a-acb0-64611b933ac9 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:37.953210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 405/806] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:37.953970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e277e8-c1f6-4e2a-acb0-64611b933ac9 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:37.959766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:38.036608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1178a-7e05-4bf2-a5db-3585e752d7f1 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.052445 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e277e8-c1f6-4e2a-acb0-64611b933ac9 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.073623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee55382-df73-4675-811b-861ba76e3973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.074859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55382-df73-4675-811b-861ba76e3973 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.075135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55382-df73-4675-811b-861ba76e3973 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.075532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee55382-df73-4675-811b-861ba76e3973 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.076125 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee55382-df73-4675-811b-861ba76e3973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:38.076433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee55382-df73-4675-811b-861ba76e3973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.077802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee55382-df73-4675-811b-861ba76e3973 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.087168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d954b-0a5a-4d1e-8737-a1fad2fb8cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.088047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d954b-0a5a-4d1e-8737-a1fad2fb8cf1 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.088322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d954b-0a5a-4d1e-8737-a1fad2fb8cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.088592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d954b-0a5a-4d1e-8737-a1fad2fb8cf1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.089239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d954b-0a5a-4d1e-8737-a1fad2fb8cf1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.089488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 402/807] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:38.090692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d954b-0a5a-4d1e-8737-a1fad2fb8cf1 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.090897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08e4ec-b0ac-4825-ace9-6df91a314702 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.091188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08e4ec-b0ac-4825-ace9-6df91a314702 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.091461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08e4ec-b0ac-4825-ace9-6df91a314702 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.093903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 403/808] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:38.093903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08e4ec-b0ac-4825-ace9-6df91a314702 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-091d954b-0a5a-4d1e-8737-a1fad2fb8cf1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:38.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:38.108302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:38.109217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 406/809] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Jul 14 15:35:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:38.110179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d954b-0a5a-4d1e-8737-a1fad2fb8cf1 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.156264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:38.183163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:38.202657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08e4ec-b0ac-4825-ace9-6df91a314702 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.209532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d954b-0a5a-4d1e-8737-a1fad2fb8cf1 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.236514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7d16c-1ca9-45b0-8034-fc5df810e21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.237177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7d16c-1ca9-45b0-8034-fc5df810e21b None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.237459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7d16c-1ca9-45b0-8034-fc5df810e21b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.237731 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7d16c-1ca9-45b0-8034-fc5df810e21b None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.238503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7d16c-1ca9-45b0-8034-fc5df810e21b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:38.238808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7d16c-1ca9-45b0-8034-fc5df810e21b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.240170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7d16c-1ca9-45b0-8034-fc5df810e21b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.242623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972a554-e415-4354-ba02-71939658f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.243498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972a554-e415-4354-ba02-71939658f043 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.243777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972a554-e415-4354-ba02-71939658f043 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.244050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972a554-e415-4354-ba02-71939658f043 None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/613c0627a67041cd858b61ecc3d08421/roles/3cb613c2ddca467295425cf3890d1dc3` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.248231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 404/810] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:38.249015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7d16c-1ca9-45b0-8034-fc5df810e21b None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.259125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972a554-e415-4354-ba02-71939658f043 None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=613c0627a67041cd858b61ecc3d08421, role_id=3cb613c2ddca467295425cf3890d1dc3)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.260544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972a554-e415-4354-ba02-71939658f043 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.346816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7d16c-1ca9-45b0-8034-fc5df810e21b None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.365388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 407/811] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:35:38 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/613c0627a67041cd858b61ecc3d08421/roles/3cb613c2ddca467295425cf3890d1dc3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:35:38.366854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63b08-ebd2-443e-87ba-796a3ff3b888 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.367230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63b08-ebd2-443e-87ba-796a3ff3b888 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.367629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63b08-ebd2-443e-87ba-796a3ff3b888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.380844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a738b-3637-4d9e-bd7c-01906890d09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.381664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a738b-3637-4d9e-bd7c-01906890d09a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.382057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a738b-3637-4d9e-bd7c-01906890d09a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.382334 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a738b-3637-4d9e-bd7c-01906890d09a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.390790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a738b-3637-4d9e-bd7c-01906890d09a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.392267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a738b-3637-4d9e-bd7c-01906890d09a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.401804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8ab63b08-ebd2-443e-87ba-796a3ff3b888 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:38.403634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8ab63b08-ebd2-443e-87ba-796a3ff3b888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-b74a738b-3637-4d9e-bd7c-01906890d09a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:38.408343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:38.409296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:38.410633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:38.410633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:38.410633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:38.410633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:38.410633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:38.410633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:38.410633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:38.410633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:38.410633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 405/812] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:38.411128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a738b-3637-4d9e-bd7c-01906890d09a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.414768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:38.483548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 408/813] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:38.484509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab63b08-ebd2-443e-87ba-796a3ff3b888 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.512441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:38.544896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a738b-3637-4d9e-bd7c-01906890d09a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.583475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5c784-4d96-48b5-a6cf-924fde351cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.584157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5c784-4d96-48b5-a6cf-924fde351cea None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.584486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5c784-4d96-48b5-a6cf-924fde351cea None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.584773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5c784-4d96-48b5-a6cf-924fde351cea None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.585580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5c784-4d96-48b5-a6cf-924fde351cea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:38.585789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5c784-4d96-48b5-a6cf-924fde351cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.587201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5c784-4d96-48b5-a6cf-924fde351cea None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.596746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 406/814] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 14 15:35:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:38.596746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5c784-4d96-48b5-a6cf-924fde351cea None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.600293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab63b08-ebd2-443e-87ba-796a3ff3b888 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.636297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce93ae-62d8-4419-8d1b-e392b3c04385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.637235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce93ae-62d8-4419-8d1b-e392b3c04385 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.637578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce93ae-62d8-4419-8d1b-e392b3c04385 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.637961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce93ae-62d8-4419-8d1b-e392b3c04385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.649716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce93ae-62d8-4419-8d1b-e392b3c04385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.651300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce93ae-62d8-4419-8d1b-e392b3c04385 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.669889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-01ce93ae-62d8-4419-8d1b-e392b3c04385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:38.670056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:38.671107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:38.672213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:38.672213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:38.672213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:38.672213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:38.672910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 409/815] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:38.673703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce93ae-62d8-4419-8d1b-e392b3c04385 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.715371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5c784-4d96-48b5-a6cf-924fde351cea None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.752215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e341e25-3ce8-4fbd-820d-91f8f87438bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.752964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341e25-3ce8-4fbd-820d-91f8f87438bb None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.753312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341e25-3ce8-4fbd-820d-91f8f87438bb None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.753653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e341e25-3ce8-4fbd-820d-91f8f87438bb None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.754547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e341e25-3ce8-4fbd-820d-91f8f87438bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:38.754919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e341e25-3ce8-4fbd-820d-91f8f87438bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.756419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e341e25-3ce8-4fbd-820d-91f8f87438bb None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.757860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:38.773844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 407/816] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:38.773844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e341e25-3ce8-4fbd-820d-91f8f87438bb None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.782013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce93ae-62d8-4419-8d1b-e392b3c04385 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.817510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cb326-59e7-44de-8e09-a37560cd01af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.818376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cb326-59e7-44de-8e09-a37560cd01af None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.818657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cb326-59e7-44de-8e09-a37560cd01af None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.818924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cb326-59e7-44de-8e09-a37560cd01af None admin] PATH_INFO: `/v3/services/network` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.819582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cb326-59e7-44de-8e09-a37560cd01af None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.820942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cb326-59e7-44de-8e09-a37560cd01af None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-353cb326-59e7-44de-8e09-a37560cd01af None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:38.834552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:38.835787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:38.836628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 410/817] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:38 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:38.837471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cb326-59e7-44de-8e09-a37560cd01af None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.898700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e341e25-3ce8-4fbd-820d-91f8f87438bb None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.930237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:38.938380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19dff-6c90-486d-aecc-4ae8c69795fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.939054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19dff-6c90-486d-aecc-4ae8c69795fe None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.939339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19dff-6c90-486d-aecc-4ae8c69795fe None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.939612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19dff-6c90-486d-aecc-4ae8c69795fe None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.940690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cb326-59e7-44de-8e09-a37560cd01af None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:38.940825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b19dff-6c90-486d-aecc-4ae8c69795fe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:38.941047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b19dff-6c90-486d-aecc-4ae8c69795fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.942456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b19dff-6c90-486d-aecc-4ae8c69795fe None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.947856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 408/818] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 14 15:35:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:38.951405 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19dff-6c90-486d-aecc-4ae8c69795fe None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:38.980533 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae093b8-50c9-41e5-a863-073bf4306762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:38.981441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae093b8-50c9-41e5-a863-073bf4306762 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:38.981728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae093b8-50c9-41e5-a863-073bf4306762 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:38.982028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae093b8-50c9-41e5-a863-073bf4306762 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:38.982751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae093b8-50c9-41e5-a863-073bf4306762 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:38.983052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae093b8-50c9-41e5-a863-073bf4306762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:38.984398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae093b8-50c9-41e5-a863-073bf4306762 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:38.995512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:38.999590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 411/819] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:39.000717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae093b8-50c9-41e5-a863-073bf4306762 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:39.065112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19dff-6c90-486d-aecc-4ae8c69795fe None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:39.101841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8174fc-c716-4384-8897-e6e0918964bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:39.102765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8174fc-c716-4384-8897-e6e0918964bd None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:39.103052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8174fc-c716-4384-8897-e6e0918964bd None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:39.103327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8174fc-c716-4384-8897-e6e0918964bd None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/95f36b4517ee4b64bc43e000c854df30/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:39.105305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8174fc-c716-4384-8897-e6e0918964bd None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=95f36b4517ee4b64bc43e000c854df30, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:39.106925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8174fc-c716-4384-8897-e6e0918964bd None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:39.125006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae093b8-50c9-41e5-a863-073bf4306762 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:39.159532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af94d47-748f-471e-9009-2d2209c37c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:39.160130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af94d47-748f-471e-9009-2d2209c37c1a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:39.160407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af94d47-748f-471e-9009-2d2209c37c1a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:39.160677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af94d47-748f-471e-9009-2d2209c37c1a None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:39.161373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af94d47-748f-471e-9009-2d2209c37c1a None admin] RBAC: Adding query filter params (name=network) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:39.161672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af94d47-748f-471e-9009-2d2209c37c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:39.163074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af94d47-748f-471e-9009-2d2209c37c1a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:39.167477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 412/820] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:35:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:39.168245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af94d47-748f-471e-9009-2d2209c37c1a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:39.185255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 409/821] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:35:38 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/95f36b4517ee4b64bc43e000c854df30/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:35:39.186494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8174fc-c716-4384-8897-e6e0918964bd None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:39.241495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:39.274206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:39.298220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af94d47-748f-471e-9009-2d2209c37c1a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:39.325085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8174fc-c716-4384-8897-e6e0918964bd None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:39.353361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe97e1d-9dfd-47e6-807f-d8ccef12a266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:39.354174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe97e1d-9dfd-47e6-807f-d8ccef12a266 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:39.354697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe97e1d-9dfd-47e6-807f-d8ccef12a266 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:39.355039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe97e1d-9dfd-47e6-807f-d8ccef12a266 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:39.356495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe97e1d-9dfd-47e6-807f-d8ccef12a266 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:39.358132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe97e1d-9dfd-47e6-807f-d8ccef12a266 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:39.366207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9b728-f35b-47dd-92db-0d76bcad8b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:39.367046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9b728-f35b-47dd-92db-0d76bcad8b1d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:39.367396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9b728-f35b-47dd-92db-0d76bcad8b1d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:39.367870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9b728-f35b-47dd-92db-0d76bcad8b1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:39.379528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9b728-f35b-47dd-92db-0d76bcad8b1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:39.381265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9b728-f35b-47dd-92db-0d76bcad8b1d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-adf9b728-f35b-47dd-92db-0d76bcad8b1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:39.397955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:39.399178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:35:39.400352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:39.405634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 410/822] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 14 15:35:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:39.405634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9b728-f35b-47dd-92db-0d76bcad8b1d None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:39.481428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 413/823] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 14 15:35:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:39.487047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe97e1d-9dfd-47e6-807f-d8ccef12a266 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:39.520549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9b728-f35b-47dd-92db-0d76bcad8b1d None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:39.537756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:39.572730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb6c2-9aa4-4321-9e71-519d9c37761c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:39.573663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb6c2-9aa4-4321-9e71-519d9c37761c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:39.574194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb6c2-9aa4-4321-9e71-519d9c37761c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:39.574654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb6c2-9aa4-4321-9e71-519d9c37761c None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:39.575530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abb6c2-9aa4-4321-9e71-519d9c37761c None admin] RBAC: Adding query filter params (type=network) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:39.575986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abb6c2-9aa4-4321-9e71-519d9c37761c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:39.579028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abb6c2-9aa4-4321-9e71-519d9c37761c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:39.586078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 411/824] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:35:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:39.591930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb6c2-9aa4-4321-9e71-519d9c37761c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:39.719040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb6c2-9aa4-4321-9e71-519d9c37761c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:39.723675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe97e1d-9dfd-47e6-807f-d8ccef12a266 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:39.762558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:39.769510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb92acc-26be-4e74-a25e-604ea71e69ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:39.770348 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92acc-26be-4e74-a25e-604ea71e69ef None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:39.770730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92acc-26be-4e74-a25e-604ea71e69ef None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:39.771236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92acc-26be-4e74-a25e-604ea71e69ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:39.771649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673f2ae-b15c-4a31-81dc-1c0a6aeffeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:39.772104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb92acc-26be-4e74-a25e-604ea71e69ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:39.772664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673f2ae-b15c-4a31-81dc-1c0a6aeffeed None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:39.773097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673f2ae-b15c-4a31-81dc-1c0a6aeffeed None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:39.773521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673f2ae-b15c-4a31-81dc-1c0a6aeffeed None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:39.773694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb92acc-26be-4e74-a25e-604ea71e69ef None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:39.774695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673f2ae-b15c-4a31-81dc-1c0a6aeffeed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:39.775176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673f2ae-b15c-4a31-81dc-1c0a6aeffeed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:39.776860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673f2ae-b15c-4a31-81dc-1c0a6aeffeed None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:39.797075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 414/825] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 14 15:35:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:39.802235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673f2ae-b15c-4a31-81dc-1c0a6aeffeed None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:39.875784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 412/826] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 14 15:35:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:39.916524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673f2ae-b15c-4a31-81dc-1c0a6aeffeed None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:39.980931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bda0b0-cb70-44b9-8bc8-6022fbf7747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:39.981919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda0b0-cb70-44b9-8bc8-6022fbf7747d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:39.982357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda0b0-cb70-44b9-8bc8-6022fbf7747d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:39.982782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda0b0-cb70-44b9-8bc8-6022fbf7747d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:40.046709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:40.078473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bda0b0-cb70-44b9-8bc8-6022fbf7747d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:40.079942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bda0b0-cb70-44b9-8bc8-6022fbf7747d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:40.185964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 415/827] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:35:40.191414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb92acc-26be-4e74-a25e-604ea71e69ef None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:40.222121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:40.304979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb92acc-26be-4e74-a25e-604ea71e69ef None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:40.341655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30343d-3bc1-4108-92c2-859639d3001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:40.342503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30343d-3bc1-4108-92c2-859639d3001d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:40.342789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30343d-3bc1-4108-92c2-859639d3001d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:40.343069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30343d-3bc1-4108-92c2-859639d3001d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:40.353851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30343d-3bc1-4108-92c2-859639d3001d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:40.355343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30343d-3bc1-4108-92c2-859639d3001d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-0d30343d-3bc1-4108-92c2-859639d3001d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:40.374878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:40.375817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:40.377335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:40.377335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:40.377335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:40.377335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:40.377335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 413/828] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:40.381734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bda0b0-cb70-44b9-8bc8-6022fbf7747d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:40.481570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72e39d-501d-4411-a01e-72dce66af93c None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:40.481867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72e39d-501d-4411-a01e-72dce66af93c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:40.482150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba72e39d-501d-4411-a01e-72dce66af93c None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:40.487967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 414/829] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:40.489562 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21e197-da98-4ce9-996d-c3f3b156649a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:40.489878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21e197-da98-4ce9-996d-c3f3b156649a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:40.490164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21e197-da98-4ce9-996d-c3f3b156649a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:40.509784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6d21e197-da98-4ce9-996d-c3f3b156649a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:40.511050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6d21e197-da98-4ce9-996d-c3f3b156649a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:40.591475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:40.619061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bda0b0-cb70-44b9-8bc8-6022fbf7747d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:40.662556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46341e8-746c-45d7-8618-a888d6238694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:40.663252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46341e8-746c-45d7-8618-a888d6238694 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:40.663538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46341e8-746c-45d7-8618-a888d6238694 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:40.663815 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46341e8-746c-45d7-8618-a888d6238694 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:40.664562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46341e8-746c-45d7-8618-a888d6238694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:40.664871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46341e8-746c-45d7-8618-a888d6238694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:40.666305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46341e8-746c-45d7-8618-a888d6238694 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:40.679862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 416/830] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:40.685711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46341e8-746c-45d7-8618-a888d6238694 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:40.728413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 415/831] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:40.749422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21e197-da98-4ce9-996d-c3f3b156649a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:40.764484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:40.802410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46341e8-746c-45d7-8618-a888d6238694 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:40.848936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5688e-7b36-4a6e-84d8-bddf4b6e9203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:40.849762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5688e-7b36-4a6e-84d8-bddf4b6e9203 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:40.850660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5688e-7b36-4a6e-84d8-bddf4b6e9203 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:40.851017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5688e-7b36-4a6e-84d8-bddf4b6e9203 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:40.853737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:40.861089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5688e-7b36-4a6e-84d8-bddf4b6e9203 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:40.863365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5688e-7b36-4a6e-84d8-bddf4b6e9203 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:40.879917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d21e197-da98-4ce9-996d-c3f3b156649a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-06c5688e-7b36-4a6e-84d8-bddf4b6e9203 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:40.882441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:40.883485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:40.884401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:40.884401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:40.884401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:40.884401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:40.884401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:40.884401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:40.884401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:40.884401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:40.884936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 417/832] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:40.890772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5688e-7b36-4a6e-84d8-bddf4b6e9203 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:40.928930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bbbb1-5da1-4446-bd87-0bab604e94a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:40.929734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bbbb1-5da1-4446-bd87-0bab604e94a7 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:40.930058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bbbb1-5da1-4446-bd87-0bab604e94a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:40.930333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bbbb1-5da1-4446-bd87-0bab604e94a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:40.942689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bbbb1-5da1-4446-bd87-0bab604e94a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-c35bbbb1-5da1-4446-bd87-0bab604e94a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:40.951144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:40.952316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:35:40.953282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:40.957352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 416/833] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:35:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:40.958266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bbbb1-5da1-4446-bd87-0bab604e94a7 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:41.028202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5688e-7b36-4a6e-84d8-bddf4b6e9203 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:41.032258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:41.073349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcb4ad-ea18-4448-8244-8b18d9ee0ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:41.074235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb4ad-ea18-4448-8244-8b18d9ee0ff3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.074619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb4ad-ea18-4448-8244-8b18d9ee0ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.074998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb4ad-ea18-4448-8244-8b18d9ee0ff3 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.075850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bbbb1-5da1-4446-bd87-0bab604e94a7 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:41.075969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb4ad-ea18-4448-8244-8b18d9ee0ff3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:41.076595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb4ad-ea18-4448-8244-8b18d9ee0ff3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:41.080145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb4ad-ea18-4448-8244-8b18d9ee0ff3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:41.097962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 418/834] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:41.102597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcb4ad-ea18-4448-8244-8b18d9ee0ff3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:41.124136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15b323-9a9b-4e40-aba5-f7ffe0aa2a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:41.125062 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15b323-9a9b-4e40-aba5-f7ffe0aa2a06 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.125346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15b323-9a9b-4e40-aba5-f7ffe0aa2a06 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.125627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15b323-9a9b-4e40-aba5-f7ffe0aa2a06 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.126918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15b323-9a9b-4e40-aba5-f7ffe0aa2a06 None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:41.127228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15b323-9a9b-4e40-aba5-f7ffe0aa2a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:41.128863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15b323-9a9b-4e40-aba5-f7ffe0aa2a06 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:41.144580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 417/835] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:35:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:41.146186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15b323-9a9b-4e40-aba5-f7ffe0aa2a06 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:41.180712 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:41.240158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcb4ad-ea18-4448-8244-8b18d9ee0ff3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:41.282047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15b323-9a9b-4e40-aba5-f7ffe0aa2a06 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:41.285952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df653c-f271-4b04-b38c-b126da21defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:41.286891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df653c-f271-4b04-b38c-b126da21defe None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.287381 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df653c-f271-4b04-b38c-b126da21defe None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.287881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df653c-f271-4b04-b38c-b126da21defe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.288920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df653c-f271-4b04-b38c-b126da21defe None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=ce88c973feef4627a1b871ee9cb26511) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:41.289487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df653c-f271-4b04-b38c-b126da21defe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:41.291268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df653c-f271-4b04-b38c-b126da21defe None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:41.306348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 419/836] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:41 2022] GET /identity/v3/role_assignments?user.id=ce88c973feef4627a1b871ee9cb26511&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=abfb60dccca7470295258595a66a5d17 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:41.315042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:41.320635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae17ac4-83fa-404c-b69d-dc7baa4674b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:41.321456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae17ac4-83fa-404c-b69d-dc7baa4674b1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.321734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae17ac4-83fa-404c-b69d-dc7baa4674b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.322037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae17ac4-83fa-404c-b69d-dc7baa4674b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.335723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae17ac4-83fa-404c-b69d-dc7baa4674b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:41.337330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae17ac4-83fa-404c-b69d-dc7baa4674b1 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-8ae17ac4-83fa-404c-b69d-dc7baa4674b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:41.357663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:41.358660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:41.359559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:41.359559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:41.359559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:41.359559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:41.366310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 418/837] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:41.368256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df653c-f271-4b04-b38c-b126da21defe None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:41.483492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df653c-f271-4b04-b38c-b126da21defe None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:41.502241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:41.520680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cdf2b-ec9b-4c07-90ba-da9749fff4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:41.521359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cdf2b-ec9b-4c07-90ba-da9749fff4bc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.521749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cdf2b-ec9b-4c07-90ba-da9749fff4bc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.522059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cdf2b-ec9b-4c07-90ba-da9749fff4bc None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.522792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cdf2b-ec9b-4c07-90ba-da9749fff4bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:41.523103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cdf2b-ec9b-4c07-90ba-da9749fff4bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:41.524465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cdf2b-ec9b-4c07-90ba-da9749fff4bc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:41.539282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 420/838] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:41.544837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae17ac4-83fa-404c-b69d-dc7baa4674b1 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:41.552947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873d946-0f73-4640-9678-e4ba070959cd None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.553267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873d946-0f73-4640-9678-e4ba070959cd None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.553497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873d946-0f73-4640-9678-e4ba070959cd None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.555204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 421/839] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:41.560842 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56f895-23e1-4a28-8010-6ba34f4af6f2 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.561174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56f895-23e1-4a28-8010-6ba34f4af6f2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.561461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56f895-23e1-4a28-8010-6ba34f4af6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.579050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-6e56f895-23e1-4a28-8010-6ba34f4af6f2 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:41.580224 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6e56f895-23e1-4a28-8010-6ba34f4af6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:41.654385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 422/840] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:35:41.669362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae17ac4-83fa-404c-b69d-dc7baa4674b1 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:41.673912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56f895-23e1-4a28-8010-6ba34f4af6f2 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:41.708671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a941944-f962-4b8e-8797-71e293bc6cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:41.709609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a941944-f962-4b8e-8797-71e293bc6cb4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.709946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a941944-f962-4b8e-8797-71e293bc6cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.710259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a941944-f962-4b8e-8797-71e293bc6cb4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.719788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a941944-f962-4b8e-8797-71e293bc6cb4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:41.720832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a941944-f962-4b8e-8797-71e293bc6cb4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-5a941944-f962-4b8e-8797-71e293bc6cb4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:41.736800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:41.738359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:41.739283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 419/841] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:35:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:41.745511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a941944-f962-4b8e-8797-71e293bc6cb4 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:41.795072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:41.798105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56f895-23e1-4a28-8010-6ba34f4af6f2 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:41.808236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:41.835604 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93636e39-5cb3-4026-bdc8-a9c394603bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:41.837357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93636e39-5cb3-4026-bdc8-a9c394603bef None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.838021 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93636e39-5cb3-4026-bdc8-a9c394603bef None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.838508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93636e39-5cb3-4026-bdc8-a9c394603bef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.851898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93636e39-5cb3-4026-bdc8-a9c394603bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:41.854680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a941944-f962-4b8e-8797-71e293bc6cb4 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-93636e39-5cb3-4026-bdc8-a9c394603bef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:41.860464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:41.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:41.862809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 423/842] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:41.867005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c0360-4523-4949-b7ac-b5fa85480e09 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.867300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c0360-4523-4949-b7ac-b5fa85480e09 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.867584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c0360-4523-4949-b7ac-b5fa85480e09 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.869151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 424/843] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:41.871368 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c0360-4523-4949-b7ac-b5fa85480e09 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:41.892985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb73aa-9c1a-4b2d-8a70-d33125d128d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:41.894010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb73aa-9c1a-4b2d-8a70-d33125d128d3 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.894523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb73aa-9c1a-4b2d-8a70-d33125d128d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.895121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb73aa-9c1a-4b2d-8a70-d33125d128d3 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.896038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb73aa-9c1a-4b2d-8a70-d33125d128d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:41.896358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb73aa-9c1a-4b2d-8a70-d33125d128d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:41.897919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb73aa-9c1a-4b2d-8a70-d33125d128d3 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:41.921960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 420/844] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 14 15:35:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:41.921960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa21ee-a547-4897-ac14-a02c9f1094d1 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:41.921960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa21ee-a547-4897-ac14-a02c9f1094d1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:41.921960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa21ee-a547-4897-ac14-a02c9f1094d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:41.937802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-84fa21ee-a547-4897-ac14-a02c9f1094d1 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:41.939309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-84fa21ee-a547-4897-ac14-a02c9f1094d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:41.984653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c0360-4523-4949-b7ac-b5fa85480e09 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.017854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 421/845] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:42.017854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa21ee-a547-4897-ac14-a02c9f1094d1 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.023502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fca654-3f7f-48fe-b0c2-d1f149a04d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.024295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fca654-3f7f-48fe-b0c2-d1f149a04d93 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.024627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fca654-3f7f-48fe-b0c2-d1f149a04d93 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.024913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fca654-3f7f-48fe-b0c2-d1f149a04d93 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.025727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fca654-3f7f-48fe-b0c2-d1f149a04d93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:42.026201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fca654-3f7f-48fe-b0c2-d1f149a04d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.028166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fca654-3f7f-48fe-b0c2-d1f149a04d93 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.042402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 425/846] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:42.042402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fca654-3f7f-48fe-b0c2-d1f149a04d93 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.053924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:42.077205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:42.127910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa21ee-a547-4897-ac14-a02c9f1094d1 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.149581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fca654-3f7f-48fe-b0c2-d1f149a04d93 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.164508 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611f2ad-6c5d-452a-b8cd-ba6914b1d4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.165456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611f2ad-6c5d-452a-b8cd-ba6914b1d4a0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.166000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611f2ad-6c5d-452a-b8cd-ba6914b1d4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.166483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611f2ad-6c5d-452a-b8cd-ba6914b1d4a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.184438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfca34-5a5c-4168-abbc-ae3f68de55e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.185176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfca34-5a5c-4168-abbc-ae3f68de55e4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.185460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfca34-5a5c-4168-abbc-ae3f68de55e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.185736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfca34-5a5c-4168-abbc-ae3f68de55e4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.186419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfca34-5a5c-4168-abbc-ae3f68de55e4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.187780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfca34-5a5c-4168-abbc-ae3f68de55e4 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-30cfca34-5a5c-4168-abbc-ae3f68de55e4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:42.200757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:42.201755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:42.202673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 426/847] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:42.203487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfca34-5a5c-4168-abbc-ae3f68de55e4 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.246774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611f2ad-6c5d-452a-b8cd-ba6914b1d4a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.248232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611f2ad-6c5d-452a-b8cd-ba6914b1d4a0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.281882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:42.302912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:42.309952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfca34-5a5c-4168-abbc-ae3f68de55e4 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.346008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7317-4f60-4e82-9944-aa40667f12de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.347048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7317-4f60-4e82-9944-aa40667f12de None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.347329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7317-4f60-4e82-9944-aa40667f12de None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.347606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d7317-4f60-4e82-9944-aa40667f12de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.357757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d7317-4f60-4e82-9944-aa40667f12de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.363790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 422/848] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:35:42.363930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d7317-4f60-4e82-9944-aa40667f12de None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.364010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611f2ad-6c5d-452a-b8cd-ba6914b1d4a0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-3b3d7317-4f60-4e82-9944-aa40667f12de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:42.381495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:42.382508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:42.383399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:42.383399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:42.383399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:42.383399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:42.387568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 427/849] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:42.388268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7317-4f60-4e82-9944-aa40667f12de None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.472110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611f2ad-6c5d-452a-b8cd-ba6914b1d4a0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.490164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d7317-4f60-4e82-9944-aa40667f12de None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.503551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:42.508785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6869a-1d9a-4795-945a-4bb6b4cc08ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.509480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6869a-1d9a-4795-945a-4bb6b4cc08ea None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.509766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6869a-1d9a-4795-945a-4bb6b4cc08ea None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.510075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6869a-1d9a-4795-945a-4bb6b4cc08ea None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.510805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6869a-1d9a-4795-945a-4bb6b4cc08ea None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:42.511116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6869a-1d9a-4795-945a-4bb6b4cc08ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.512703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6869a-1d9a-4795-945a-4bb6b4cc08ea None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.517719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 423/850] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:42.518661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6869a-1d9a-4795-945a-4bb6b4cc08ea None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.534323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c5661-785d-41ee-8e17-c66ac32929c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.535126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c5661-785d-41ee-8e17-c66ac32929c8 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.535410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c5661-785d-41ee-8e17-c66ac32929c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.535684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c5661-785d-41ee-8e17-c66ac32929c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.546954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c5661-785d-41ee-8e17-c66ac32929c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.548411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c5661-785d-41ee-8e17-c66ac32929c8 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.555924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-091c5661-785d-41ee-8e17-c66ac32929c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:42.568656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:42.569837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:42.575978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:42.575978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:42.575978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:42.575978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:42.575978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 428/851] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:42.575978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c5661-785d-41ee-8e17-c66ac32929c8 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.629514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6869a-1d9a-4795-945a-4bb6b4cc08ea None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.664666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d677eb-f8e3-4d37-aa64-22a8cbb913cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.665546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d677eb-f8e3-4d37-aa64-22a8cbb913cd None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.666041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d677eb-f8e3-4d37-aa64-22a8cbb913cd None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.666561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d677eb-f8e3-4d37-aa64-22a8cbb913cd None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.667557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d677eb-f8e3-4d37-aa64-22a8cbb913cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:42.668396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d677eb-f8e3-4d37-aa64-22a8cbb913cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.669765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d677eb-f8e3-4d37-aa64-22a8cbb913cd None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.676282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c5661-785d-41ee-8e17-c66ac32929c8 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.686830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 424/852] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:42.686830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d677eb-f8e3-4d37-aa64-22a8cbb913cd None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.714374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec828-078c-433c-af5c-ca916119d236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.715102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec828-078c-433c-af5c-ca916119d236 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.715388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec828-078c-433c-af5c-ca916119d236 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.715662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec828-078c-433c-af5c-ca916119d236 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.716365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec828-078c-433c-af5c-ca916119d236 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:42.716668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec828-078c-433c-af5c-ca916119d236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.718039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec828-078c-433c-af5c-ca916119d236 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.722634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 429/853] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:42.724106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec828-078c-433c-af5c-ca916119d236 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.748440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:42.802541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d677eb-f8e3-4d37-aa64-22a8cbb913cd None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.821540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:42.852192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49016fa9-f303-489d-8410-709aaf11a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.853229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49016fa9-f303-489d-8410-709aaf11a13c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.853702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49016fa9-f303-489d-8410-709aaf11a13c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.854227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49016fa9-f303-489d-8410-709aaf11a13c None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.855186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49016fa9-f303-489d-8410-709aaf11a13c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:42.855761 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49016fa9-f303-489d-8410-709aaf11a13c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.857382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49016fa9-f303-489d-8410-709aaf11a13c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.871310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 425/854] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:42.878040 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec828-078c-433c-af5c-ca916119d236 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:42.878227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49016fa9-f303-489d-8410-709aaf11a13c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:42.924597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9580a-e7f7-4ff2-bf72-cc8e0a0621c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:42.925680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9580a-e7f7-4ff2-bf72-cc8e0a0621c6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:42.926124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9580a-e7f7-4ff2-bf72-cc8e0a0621c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:42.926980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9580a-e7f7-4ff2-bf72-cc8e0a0621c6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:42.937225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9580a-e7f7-4ff2-bf72-cc8e0a0621c6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:42.939331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9580a-e7f7-4ff2-bf72-cc8e0a0621c6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-2da9580a-e7f7-4ff2-bf72-cc8e0a0621c6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:42.973438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:42.974684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 15:35:42.975886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:42.977938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 430/855] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:42.988535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9580a-e7f7-4ff2-bf72-cc8e0a0621c6 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:43.033733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49016fa9-f303-489d-8410-709aaf11a13c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:43.063822 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:43.083937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96cd66-5039-4357-b8e0-e56c6155affa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:43.085372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96cd66-5039-4357-b8e0-e56c6155affa None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:43.085861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96cd66-5039-4357-b8e0-e56c6155affa None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:43.086289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96cd66-5039-4357-b8e0-e56c6155affa None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:43.107188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96cd66-5039-4357-b8e0-e56c6155affa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:43.108968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96cd66-5039-4357-b8e0-e56c6155affa None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-1c96cd66-5039-4357-b8e0-e56c6155affa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:43.133324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:43.134328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:43.135221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:43.135221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:43.135221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:43.135221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:43.135221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:43.135221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:43.135221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:43.135221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:43.149436 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 426/856] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:43.158982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96cd66-5039-4357-b8e0-e56c6155affa None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:43.245637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9580a-e7f7-4ff2-bf72-cc8e0a0621c6 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:43.292669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:43.315282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d094834-c12a-4719-bf69-975704a06f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:43.316315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d094834-c12a-4719-bf69-975704a06f6d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:43.317727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d094834-c12a-4719-bf69-975704a06f6d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:43.318884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d094834-c12a-4719-bf69-975704a06f6d None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:43.319812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d094834-c12a-4719-bf69-975704a06f6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:43.321662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d094834-c12a-4719-bf69-975704a06f6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:43.323237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d094834-c12a-4719-bf69-975704a06f6d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:43.335357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96cd66-5039-4357-b8e0-e56c6155affa None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:43.365662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 431/857] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 14 15:35:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:43.371054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d094834-c12a-4719-bf69-975704a06f6d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:43.398199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b8e9c-9b54-4f00-baf4-c8d4afebb4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:43.399227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b8e9c-9b54-4f00-baf4-c8d4afebb4d2 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:43.399742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b8e9c-9b54-4f00-baf4-c8d4afebb4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:43.400171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b8e9c-9b54-4f00-baf4-c8d4afebb4d2 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:43.401185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b8e9c-9b54-4f00-baf4-c8d4afebb4d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:43.401682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b8e9c-9b54-4f00-baf4-c8d4afebb4d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:43.403727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29b8e9c-9b54-4f00-baf4-c8d4afebb4d2 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:43.422672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 427/858] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:43.423885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b8e9c-9b54-4f00-baf4-c8d4afebb4d2 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:43.463611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:43.478811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d094834-c12a-4719-bf69-975704a06f6d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:43.518103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235896f4-f324-4369-80e7-fbe9e9cf6cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:43.518795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235896f4-f324-4369-80e7-fbe9e9cf6cb6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:43.519083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235896f4-f324-4369-80e7-fbe9e9cf6cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:43.519359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235896f4-f324-4369-80e7-fbe9e9cf6cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:43.534584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b8e9c-9b54-4f00-baf4-c8d4afebb4d2 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:43.568398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23fb436-6dd7-4a9c-b8a2-d92885093d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:43.569047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fb436-6dd7-4a9c-b8a2-d92885093d66 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:43.569327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fb436-6dd7-4a9c-b8a2-d92885093d66 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:43.569609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23fb436-6dd7-4a9c-b8a2-d92885093d66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:43.570456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fb436-6dd7-4a9c-b8a2-d92885093d66 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=613c0627a67041cd858b61ecc3d08421) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:43.570760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fb436-6dd7-4a9c-b8a2-d92885093d66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:43.571422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:43.572183 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23fb436-6dd7-4a9c-b8a2-d92885093d66 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:43.585862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 428/859] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:43 2022] GET /identity/v3/role_assignments?user.id=613c0627a67041cd858b61ecc3d08421&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:43.585862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ca269-dfe8-42ec-8f78-87ff12f90e7e None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:43.585862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ca269-dfe8-42ec-8f78-87ff12f90e7e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:43.585862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ca269-dfe8-42ec-8f78-87ff12f90e7e None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:43.585862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 429/860] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:43.587532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa4f72-95b1-4c56-9101-4f02ec908988 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:43.587814 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa4f72-95b1-4c56-9101-4f02ec908988 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:43.588089 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa4f72-95b1-4c56-9101-4f02ec908988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:43.605876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235896f4-f324-4369-80e7-fbe9e9cf6cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:43.607732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235896f4-f324-4369-80e7-fbe9e9cf6cb6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:43.608011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5daa4f72-95b1-4c56-9101-4f02ec908988 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:43.609292 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5daa4f72-95b1-4c56-9101-4f02ec908988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:43.697845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 430/861] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:43.712677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa4f72-95b1-4c56-9101-4f02ec908988 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:43.733626 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 432/862] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 15:35:43.739187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235896f4-f324-4369-80e7-fbe9e9cf6cb6 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:43.756240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:43.835808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:43.841741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa4f72-95b1-4c56-9101-4f02ec908988 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:43.863560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235896f4-f324-4369-80e7-fbe9e9cf6cb6 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:43.889209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fcb42-03a5-47c7-ba7d-dc45f1764d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:43.890934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fcb42-03a5-47c7-ba7d-dc45f1764d2c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:43.890934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fcb42-03a5-47c7-ba7d-dc45f1764d2c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:43.890934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fcb42-03a5-47c7-ba7d-dc45f1764d2c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:43.891272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fcb42-03a5-47c7-ba7d-dc45f1764d2c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:43.892900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fcb42-03a5-47c7-ba7d-dc45f1764d2c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-f41fcb42-03a5-47c7-ba7d-dc45f1764d2c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:43.907823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:43.908823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:43.909688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 431/863] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Jul 14 15:35:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:43.915361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fcb42-03a5-47c7-ba7d-dc45f1764d2c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:43.916592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8768e-93d5-4984-a880-deb922e5c660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:43.917651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8768e-93d5-4984-a880-deb922e5c660 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:43.918496 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8768e-93d5-4984-a880-deb922e5c660 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:43.919987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8768e-93d5-4984-a880-deb922e5c660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:43.933373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8768e-93d5-4984-a880-deb922e5c660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:43.936382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8768e-93d5-4984-a880-deb922e5c660 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-cba8768e-93d5-4984-a880-deb922e5c660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:43.961288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:43.962257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:43.963155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:43.963155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:43.963155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:43.963155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:43.966062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 433/864] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:43.971197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8768e-93d5-4984-a880-deb922e5c660 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:44.032511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:44.051184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fcb42-03a5-47c7-ba7d-dc45f1764d2c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:44.098811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8768e-93d5-4984-a880-deb922e5c660 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:44.103023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225afef-9476-422a-8c87-8ff161822b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:44.103698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225afef-9476-422a-8c87-8ff161822b7a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:44.103980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225afef-9476-422a-8c87-8ff161822b7a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:44.104256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225afef-9476-422a-8c87-8ff161822b7a None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:44.104978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225afef-9476-422a-8c87-8ff161822b7a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:44.105802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225afef-9476-422a-8c87-8ff161822b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:44.107224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225afef-9476-422a-8c87-8ff161822b7a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:44.114451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 432/865] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:44.121077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225afef-9476-422a-8c87-8ff161822b7a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:44.142878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:44.148889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894826-b752-4d77-a47b-b9c1c60ee9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:44.152998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894826-b752-4d77-a47b-b9c1c60ee9cd None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:44.152998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894826-b752-4d77-a47b-b9c1c60ee9cd None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:44.152998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894826-b752-4d77-a47b-b9c1c60ee9cd None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:44.152998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc894826-b752-4d77-a47b-b9c1c60ee9cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:44.152998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc894826-b752-4d77-a47b-b9c1c60ee9cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:44.153700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc894826-b752-4d77-a47b-b9c1c60ee9cd None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:44.165925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 434/866] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:44.172837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894826-b752-4d77-a47b-b9c1c60ee9cd None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:44.235373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225afef-9476-422a-8c87-8ff161822b7a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:44.270378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b930f66-c105-405e-ae79-31f4d45c9ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:44.271047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b930f66-c105-405e-ae79-31f4d45c9ed6 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:44.271330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b930f66-c105-405e-ae79-31f4d45c9ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:44.271602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b930f66-c105-405e-ae79-31f4d45c9ed6 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:44.272304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b930f66-c105-405e-ae79-31f4d45c9ed6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:44.272604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b930f66-c105-405e-ae79-31f4d45c9ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:44.273983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b930f66-c105-405e-ae79-31f4d45c9ed6 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:44.278922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 433/867] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:44.292586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894826-b752-4d77-a47b-b9c1c60ee9cd None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:44.331232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55af6f1-9165-412e-bc07-3b7d3524955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:44.332093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55af6f1-9165-412e-bc07-3b7d3524955a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:44.332367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55af6f1-9165-412e-bc07-3b7d3524955a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:44.332680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55af6f1-9165-412e-bc07-3b7d3524955a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:44.340780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55af6f1-9165-412e-bc07-3b7d3524955a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:44.342294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55af6f1-9165-412e-bc07-3b7d3524955a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-b55af6f1-9165-412e-bc07-3b7d3524955a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:44.358960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:44.361109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:44.363348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:44.363348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:44.363348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:44.363348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:44.363348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:44.363348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:44.363348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:44.363348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:44.363348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 435/868] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:44.369858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b930f66-c105-405e-ae79-31f4d45c9ed6 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:44.395877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:44.479085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b930f66-c105-405e-ae79-31f4d45c9ed6 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:44.514939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c84cc-a480-4c34-9496-7f59c0ddd873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:44.515603 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c84cc-a480-4c34-9496-7f59c0ddd873 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:44.515886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c84cc-a480-4c34-9496-7f59c0ddd873 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:44.516158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c84cc-a480-4c34-9496-7f59c0ddd873 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:44.516942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c84cc-a480-4c34-9496-7f59c0ddd873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:44.517246 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c84cc-a480-4c34-9496-7f59c0ddd873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:44.518697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c84cc-a480-4c34-9496-7f59c0ddd873 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:44.532313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 434/869] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:44.538295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55af6f1-9165-412e-bc07-3b7d3524955a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:44.595108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:44.643487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55af6f1-9165-412e-bc07-3b7d3524955a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:44.679032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf00ec-1ebf-43fb-83a9-f7c91374b994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:44.679707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf00ec-1ebf-43fb-83a9-f7c91374b994 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:44.679998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf00ec-1ebf-43fb-83a9-f7c91374b994 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:44.680284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf00ec-1ebf-43fb-83a9-f7c91374b994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:44.681122 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf00ec-1ebf-43fb-83a9-f7c91374b994 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=95f36b4517ee4b64bc43e000c854df30) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:44.681433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf00ec-1ebf-43fb-83a9-f7c91374b994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:44.682917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf00ec-1ebf-43fb-83a9-f7c91374b994 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:44.690095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 436/870] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:44 2022] GET /identity/v3/role_assignments?user.id=95f36b4517ee4b64bc43e000c854df30&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=abfb60dccca7470295258595a66a5d17 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:44.851505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef552c-c29a-47c7-b74f-ccd9ade2b0ce None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:44.852212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef552c-c29a-47c7-b74f-ccd9ade2b0ce None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:44.852212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef552c-c29a-47c7-b74f-ccd9ade2b0ce None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:44.853672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 435/871] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:44.860157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287402-0292-4ea0-8689-3581f415fbba None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:44.860458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287402-0292-4ea0-8689-3581f415fbba None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:44.860746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287402-0292-4ea0-8689-3581f415fbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:44.877695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e5287402-0292-4ea0-8689-3581f415fbba None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:44.878962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e5287402-0292-4ea0-8689-3581f415fbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:44.953757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 437/872] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:44.976041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef552c-c29a-47c7-b74f-ccd9ade2b0ce None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:45.066277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:45.086331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef552c-c29a-47c7-b74f-ccd9ade2b0ce None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:45.124420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c95c7d-f05f-4c98-a378-8c87fd171557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:45.125099 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c95c7d-f05f-4c98-a378-8c87fd171557 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:45.125380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c95c7d-f05f-4c98-a378-8c87fd171557 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:45.125654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c95c7d-f05f-4c98-a378-8c87fd171557 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:45.126428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c95c7d-f05f-4c98-a378-8c87fd171557 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:45.127804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c95c7d-f05f-4c98-a378-8c87fd171557 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:45.210274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 436/873] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:35:44 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:45.787396 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431a7c-d957-460e-a147-1ddc681e87b9 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:45.787913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431a7c-d957-460e-a147-1ddc681e87b9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:45.787992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431a7c-d957-460e-a147-1ddc681e87b9 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:45.789543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 438/874] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:45.795491 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c32bf-7671-4da7-899c-97394ff01d5b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:45.795794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c32bf-7671-4da7-899c-97394ff01d5b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:45.796073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c32bf-7671-4da7-899c-97394ff01d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:45.813283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-787c32bf-7671-4da7-899c-97394ff01d5b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:45.814510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-787c32bf-7671-4da7-899c-97394ff01d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:45.894483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 437/875] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:45 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:45.918859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431a7c-d957-460e-a147-1ddc681e87b9 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:45.971607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:46.028313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431a7c-d957-460e-a147-1ddc681e87b9 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:46.066817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e74de5-b8be-42a0-9956-1ea4cae070be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:46.067642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e74de5-b8be-42a0-9956-1ea4cae070be None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.067925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e74de5-b8be-42a0-9956-1ea4cae070be None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.068200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e74de5-b8be-42a0-9956-1ea4cae070be None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.079045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e74de5-b8be-42a0-9956-1ea4cae070be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-08e74de5-b8be-42a0-9956-1ea4cae070be None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:46.087486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:46.088407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:46.090306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 439/876] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 14 15:35:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:46.095597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c32bf-7671-4da7-899c-97394ff01d5b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:46.166110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:46.203304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c32bf-7671-4da7-899c-97394ff01d5b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:46.240477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837fed3-c873-4a6a-9081-7771bf98e351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:46.241155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837fed3-c873-4a6a-9081-7771bf98e351 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.241439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837fed3-c873-4a6a-9081-7771bf98e351 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.241716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837fed3-c873-4a6a-9081-7771bf98e351 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.242506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837fed3-c873-4a6a-9081-7771bf98e351 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:46.242816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837fed3-c873-4a6a-9081-7771bf98e351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:46.244196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837fed3-c873-4a6a-9081-7771bf98e351 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:46.252939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 438/877] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 14 15:35:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:46.258536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e74de5-b8be-42a0-9956-1ea4cae070be None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:46.365903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e74de5-b8be-42a0-9956-1ea4cae070be None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:46.373051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:46.401268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504fd2a-6c2d-4d26-905d-fad926ee4701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:46.402196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504fd2a-6c2d-4d26-905d-fad926ee4701 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.402489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504fd2a-6c2d-4d26-905d-fad926ee4701 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.402771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504fd2a-6c2d-4d26-905d-fad926ee4701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.413801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504fd2a-6c2d-4d26-905d-fad926ee4701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:46.415283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c504fd2a-6c2d-4d26-905d-fad926ee4701 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-c504fd2a-6c2d-4d26-905d-fad926ee4701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:46.434516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:46.437252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:46.439535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:46.439535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:46.439535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:46.439535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:46.439535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 440/878] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:46.445001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837fed3-c873-4a6a-9081-7771bf98e351 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:46.445228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5713-7584-4738-b6f8-ed22398a54ef None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.445228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5713-7584-4738-b6f8-ed22398a54ef None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.445228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab5713-7584-4738-b6f8-ed22398a54ef None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.449968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 441/879] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:46.451817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aedce-8f49-4edc-9c15-9d392ba5d2c8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.452326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aedce-8f49-4edc-9c15-9d392ba5d2c8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.452841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aedce-8f49-4edc-9c15-9d392ba5d2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.469708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-375aedce-8f49-4edc-9c15-9d392ba5d2c8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:46.471249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-375aedce-8f49-4edc-9c15-9d392ba5d2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:46.549939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 442/880] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:46 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:35:46.564080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837fed3-c873-4a6a-9081-7771bf98e351 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:46.567530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aedce-8f49-4edc-9c15-9d392ba5d2c8 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:46.600220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:46.607623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672147be-e2dd-4c6c-8407-4c8ff0a44962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:46.612660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672147be-e2dd-4c6c-8407-4c8ff0a44962 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.612660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672147be-e2dd-4c6c-8407-4c8ff0a44962 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.612660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672147be-e2dd-4c6c-8407-4c8ff0a44962 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.612660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672147be-e2dd-4c6c-8407-4c8ff0a44962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:46.612660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672147be-e2dd-4c6c-8407-4c8ff0a44962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:46.612660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672147be-e2dd-4c6c-8407-4c8ff0a44962 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:46.631361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 439/881] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:46.632557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672147be-e2dd-4c6c-8407-4c8ff0a44962 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:46.677155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aedce-8f49-4edc-9c15-9d392ba5d2c8 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:46.694231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:46.716638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83116f13-cb1a-4bfb-9d31-e9c9186fc185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:46.717473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83116f13-cb1a-4bfb-9d31-e9c9186fc185 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.717755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83116f13-cb1a-4bfb-9d31-e9c9186fc185 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.718079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83116f13-cb1a-4bfb-9d31-e9c9186fc185 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.718995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83116f13-cb1a-4bfb-9d31-e9c9186fc185 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:46.720504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83116f13-cb1a-4bfb-9d31-e9c9186fc185 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:46.742555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672147be-e2dd-4c6c-8407-4c8ff0a44962 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:46.778820 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993daff-59a3-4675-8014-af84885396ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:46.779635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993daff-59a3-4675-8014-af84885396ae None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.779923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993daff-59a3-4675-8014-af84885396ae None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.780201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993daff-59a3-4675-8014-af84885396ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.788472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993daff-59a3-4675-8014-af84885396ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:46.789956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993daff-59a3-4675-8014-af84885396ae None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-8993daff-59a3-4675-8014-af84885396ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:46.806473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:46.807526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:46.808534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:46.810019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 440/882] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:35:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:46.813456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993daff-59a3-4675-8014-af84885396ae None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:46.827852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 443/883] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:35:46 2022] POST /identity/v3/services => generated 269 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:46.876135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:46.914176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1641e71-9cf2-445e-a9a2-3a324ecfde2d None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.914496 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1641e71-9cf2-445e-a9a2-3a324ecfde2d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.914701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1641e71-9cf2-445e-a9a2-3a324ecfde2d None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.916423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 444/884] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:46.928115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b432b6-b342-478c-95e9-0a342ec78489 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.928486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b432b6-b342-478c-95e9-0a342ec78489 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.928840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b432b6-b342-478c-95e9-0a342ec78489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.940021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993daff-59a3-4675-8014-af84885396ae None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:46.952483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-44b432b6-b342-478c-95e9-0a342ec78489 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:46.954385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-44b432b6-b342-478c-95e9-0a342ec78489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:46.992471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affd44d-d024-43ca-b42b-0711afd67001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:46.993266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affd44d-d024-43ca-b42b-0711afd67001 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:46.993549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affd44d-d024-43ca-b42b-0711afd67001 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:46.993848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affd44d-d024-43ca-b42b-0711afd67001 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:46.994716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affd44d-d024-43ca-b42b-0711afd67001 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:46.995030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affd44d-d024-43ca-b42b-0711afd67001 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:46.997112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affd44d-d024-43ca-b42b-0711afd67001 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:47.016856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 441/885] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 14 15:35:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:47.024359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affd44d-d024-43ca-b42b-0711afd67001 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.037901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 445/886] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:35:47.063903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b432b6-b342-478c-95e9-0a342ec78489 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.160230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:47.168680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affd44d-d024-43ca-b42b-0711afd67001 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:47.171891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:47.184631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b432b6-b342-478c-95e9-0a342ec78489 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:47.219650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c1cda-7958-4a4c-81ac-ce42bb627a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:47.220316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c1cda-7958-4a4c-81ac-ce42bb627a70 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.220602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c1cda-7958-4a4c-81ac-ce42bb627a70 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.220879 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c1cda-7958-4a4c-81ac-ce42bb627a70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.227029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706e1a-cacf-4070-8b64-d3e7b815a59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:47.228159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706e1a-cacf-4070-8b64-d3e7b815a59c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.228671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706e1a-cacf-4070-8b64-d3e7b815a59c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.229164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706e1a-cacf-4070-8b64-d3e7b815a59c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.230117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f706e1a-cacf-4070-8b64-d3e7b815a59c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:47.231839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f706e1a-cacf-4070-8b64-d3e7b815a59c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-9f706e1a-cacf-4070-8b64-d3e7b815a59c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:47.245643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:47.246728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:47.247575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:47.250536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 446/887] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:47.256025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706e1a-cacf-4070-8b64-d3e7b815a59c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.307548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c1cda-7958-4a4c-81ac-ce42bb627a70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:47.310384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c1cda-7958-4a4c-81ac-ce42bb627a70 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:47.365195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706e1a-cacf-4070-8b64-d3e7b815a59c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:47.401142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c729808-c3cc-4d0c-aba3-b8aa5b90fdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:47.401875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c729808-c3cc-4d0c-aba3-b8aa5b90fdf3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.402649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c729808-c3cc-4d0c-aba3-b8aa5b90fdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.402927 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c729808-c3cc-4d0c-aba3-b8aa5b90fdf3 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.403638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c729808-c3cc-4d0c-aba3-b8aa5b90fdf3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:47.403948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c729808-c3cc-4d0c-aba3-b8aa5b90fdf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:47.405310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c729808-c3cc-4d0c-aba3-b8aa5b90fdf3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:47.410399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 447/888] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:47.411708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef3f3f-2225-434f-84c3-69ac95d0713a None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.411999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef3f3f-2225-434f-84c3-69ac95d0713a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.412307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef3f3f-2225-434f-84c3-69ac95d0713a None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.413516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 448/889] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:47.415649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef3f3f-2225-434f-84c3-69ac95d0713a None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.426209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 442/890] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 15:35:47.427697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56c75f-8d5d-411b-8cdd-5de1bb3b806d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.428125 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56c75f-8d5d-411b-8cdd-5de1bb3b806d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.428536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56c75f-8d5d-411b-8cdd-5de1bb3b806d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.432781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:47.435579 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:47.446607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ba56c75f-8d5d-411b-8cdd-5de1bb3b806d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:47.447805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ba56c75f-8d5d-411b-8cdd-5de1bb3b806d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:47.518089 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 443/891] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:47.518871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56c75f-8d5d-411b-8cdd-5de1bb3b806d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.528440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef3f3f-2225-434f-84c3-69ac95d0713a None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:47.566319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155f4f-0954-4e31-a43e-cbf8b05a4ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:47.566993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155f4f-0954-4e31-a43e-cbf8b05a4ae9 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.567276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155f4f-0954-4e31-a43e-cbf8b05a4ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.567555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3155f4f-0954-4e31-a43e-cbf8b05a4ae9 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.568266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3155f4f-0954-4e31-a43e-cbf8b05a4ae9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:47.568573 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3155f4f-0954-4e31-a43e-cbf8b05a4ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:47.569950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3155f4f-0954-4e31-a43e-cbf8b05a4ae9 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:47.575015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 449/892] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:47.575808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155f4f-0954-4e31-a43e-cbf8b05a4ae9 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.654538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56c75f-8d5d-411b-8cdd-5de1bb3b806d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:47.658699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:47.695835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9df59-44b9-4cca-b46e-f356d816ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:47.696578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9df59-44b9-4cca-b46e-f356d816ea4f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.696867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9df59-44b9-4cca-b46e-f356d816ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.697146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9df59-44b9-4cca-b46e-f356d816ea4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.699675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3155f4f-0954-4e31-a43e-cbf8b05a4ae9 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:47.703677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:47.717272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9df59-44b9-4cca-b46e-f356d816ea4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:47.718819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9df59-44b9-4cca-b46e-f356d816ea4f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-59e9df59-44b9-4cca-b46e-f356d816ea4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:47.738379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:47.739555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:47.740526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:47.740526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:47.740526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:47.740526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:47.740844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 444/893] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:47.747029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9df59-44b9-4cca-b46e-f356d816ea4f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.748309 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14f73e-e005-4b31-9621-abc546fbe582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:47.749283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14f73e-e005-4b31-9621-abc546fbe582 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.749938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14f73e-e005-4b31-9621-abc546fbe582 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.750370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14f73e-e005-4b31-9621-abc546fbe582 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.752068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14f73e-e005-4b31-9621-abc546fbe582 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:47.753861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14f73e-e005-4b31-9621-abc546fbe582 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-bf14f73e-e005-4b31-9621-abc546fbe582 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:47.769389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:47.770527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:47.772430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 450/894] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:47.778079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14f73e-e005-4b31-9621-abc546fbe582 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.860318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9df59-44b9-4cca-b46e-f356d816ea4f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:47.899579 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e456290-ad1d-4ecb-83e1-f8cfe0c53fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:47.900254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e456290-ad1d-4ecb-83e1-f8cfe0c53fd5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.900540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e456290-ad1d-4ecb-83e1-f8cfe0c53fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.900819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e456290-ad1d-4ecb-83e1-f8cfe0c53fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.901552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e456290-ad1d-4ecb-83e1-f8cfe0c53fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:47.901883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e456290-ad1d-4ecb-83e1-f8cfe0c53fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:47.903395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e456290-ad1d-4ecb-83e1-f8cfe0c53fd5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:47.914870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14f73e-e005-4b31-9621-abc546fbe582 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:47.915444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 445/895] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:47.922217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e456290-ad1d-4ecb-83e1-f8cfe0c53fd5 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.955603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa70d9-49fb-4f95-9595-de964b63e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:47.956430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa70d9-49fb-4f95-9595-de964b63e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:47.956816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa70d9-49fb-4f95-9595-de964b63e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:47.957200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa70d9-49fb-4f95-9595-de964b63e5c5 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:47.958065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa70d9-49fb-4f95-9595-de964b63e5c5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:47.958475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa70d9-49fb-4f95-9595-de964b63e5c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:47.959940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa70d9-49fb-4f95-9595-de964b63e5c5 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:47.961137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:47.966443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 451/896] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:47.971953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa70d9-49fb-4f95-9595-de964b63e5c5 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:47.989109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:48.050867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e456290-ad1d-4ecb-83e1-f8cfe0c53fd5 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:48.097553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa70d9-49fb-4f95-9595-de964b63e5c5 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:48.110465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64884bf-97a5-4524-925f-018d8ff94087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:48.111959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64884bf-97a5-4524-925f-018d8ff94087 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:48.112770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64884bf-97a5-4524-925f-018d8ff94087 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:48.113234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64884bf-97a5-4524-925f-018d8ff94087 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:48.123513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64884bf-97a5-4524-925f-018d8ff94087 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:48.126040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64884bf-97a5-4524-925f-018d8ff94087 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:48.137257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400890db-92d8-4ecf-ab20-cf6012e906aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:48.138056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400890db-92d8-4ecf-ab20-cf6012e906aa None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:48.138411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400890db-92d8-4ecf-ab20-cf6012e906aa None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:48.138756 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400890db-92d8-4ecf-ab20-cf6012e906aa None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:48.139538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400890db-92d8-4ecf-ab20-cf6012e906aa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:48.139932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400890db-92d8-4ecf-ab20-cf6012e906aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:48.141392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400890db-92d8-4ecf-ab20-cf6012e906aa None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-e64884bf-97a5-4524-925f-018d8ff94087 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:48.146817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:48.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:48.148600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:48.148600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:48.148600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:48.148600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:48.148600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:48.148600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:48.148600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:48.148600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:48.155435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 452/898] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:48.155435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400890db-92d8-4ecf-ab20-cf6012e906aa None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:48.155650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 446/898] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:48.158405 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64884bf-97a5-4524-925f-018d8ff94087 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:48.225259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:48.266121 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400890db-92d8-4ecf-ab20-cf6012e906aa None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:48.275004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64884bf-97a5-4524-925f-018d8ff94087 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:48.284223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:48.302320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95be04d-6318-495e-a9a5-14eb86c4952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:48.303290 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be04d-6318-495e-a9a5-14eb86c4952b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:48.303804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be04d-6318-495e-a9a5-14eb86c4952b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:48.304294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95be04d-6318-495e-a9a5-14eb86c4952b None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:48.305282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95be04d-6318-495e-a9a5-14eb86c4952b None admin] RBAC: Adding query filter params (interface=public, service_id=39f5f5f56bcb48dbb6cd9a6638ada54d, region_id=RegionOne) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:48.305805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95be04d-6318-495e-a9a5-14eb86c4952b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:48.307376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95be04d-6318-495e-a9a5-14eb86c4952b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:48.314526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 453/899] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 14 15:35:48 2022] GET /identity/v3/endpoints?service_id=39f5f5f56bcb48dbb6cd9a6638ada54d&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:48.319768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dc355-3f5d-444a-bc9f-5797afa39e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:48.320107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95be04d-6318-495e-a9a5-14eb86c4952b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:48.320937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dc355-3f5d-444a-bc9f-5797afa39e74 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:48.321423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dc355-3f5d-444a-bc9f-5797afa39e74 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:48.321954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982dc355-3f5d-444a-bc9f-5797afa39e74 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:48.323167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982dc355-3f5d-444a-bc9f-5797afa39e74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:48.324619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982dc355-3f5d-444a-bc9f-5797afa39e74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:48.326408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982dc355-3f5d-444a-bc9f-5797afa39e74 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:48.347544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 447/900] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:48.348600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dc355-3f5d-444a-bc9f-5797afa39e74 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:48.445610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95be04d-6318-495e-a9a5-14eb86c4952b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:48.473317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:48.484495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a80ef-8112-4e88-8c72-adf97b375573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:48.484495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a80ef-8112-4e88-8c72-adf97b375573 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:48.484495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a80ef-8112-4e88-8c72-adf97b375573 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:48.484495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a80ef-8112-4e88-8c72-adf97b375573 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:48.484857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a80ef-8112-4e88-8c72-adf97b375573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:48.486589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a80ef-8112-4e88-8c72-adf97b375573 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:48.490376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982dc355-3f5d-444a-bc9f-5797afa39e74 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:48.498282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 454/901] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 14 15:35:48 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:35:48.531747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765f32d-3201-4a09-8911-3c5c4f7ead1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:48.532435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765f32d-3201-4a09-8911-3c5c4f7ead1f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:48.532720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765f32d-3201-4a09-8911-3c5c4f7ead1f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:48.533000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765f32d-3201-4a09-8911-3c5c4f7ead1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:48.533890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765f32d-3201-4a09-8911-3c5c4f7ead1f None admin] RBAC: Adding query filter params (role.id=5ccecebd4103442484b3dadfbfbf47c4, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=613c0627a67041cd858b61ecc3d08421) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:48.534214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765f32d-3201-4a09-8911-3c5c4f7ead1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:48.535676 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765f32d-3201-4a09-8911-3c5c4f7ead1f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:48.542590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 448/902] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:48 2022] GET /identity/v3/role_assignments?user.id=613c0627a67041cd858b61ecc3d08421&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=5ccecebd4103442484b3dadfbfbf47c4 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:49.002661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12303c9a-09ad-4486-805c-670001e4307c None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.003825 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12303c9a-09ad-4486-805c-670001e4307c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.004111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12303c9a-09ad-4486-805c-670001e4307c None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.005637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 455/903] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:49.011162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfad51-4a94-49b9-a39d-0005f83fe68d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.012071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfad51-4a94-49b9-a39d-0005f83fe68d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.012578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfad51-4a94-49b9-a39d-0005f83fe68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.030889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-74dfad51-4a94-49b9-a39d-0005f83fe68d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:49.032385 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-74dfad51-4a94-49b9-a39d-0005f83fe68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:49.061080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:49.106828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 449/904] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:35:49.130086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12303c9a-09ad-4486-805c-670001e4307c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:49.244670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12303c9a-09ad-4486-805c-670001e4307c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:49.281061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecba0f-94d8-41db-9b1a-82baa4e9662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:49.281806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecba0f-94d8-41db-9b1a-82baa4e9662b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.282118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecba0f-94d8-41db-9b1a-82baa4e9662b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.282406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ecba0f-94d8-41db-9b1a-82baa4e9662b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.283069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecba0f-94d8-41db-9b1a-82baa4e9662b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:49.284438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ecba0f-94d8-41db-9b1a-82baa4e9662b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-23ecba0f-94d8-41db-9b1a-82baa4e9662b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:49.297085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:49.298098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:49.298939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 456/905] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Jul 14 15:35:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:49.306897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfad51-4a94-49b9-a39d-0005f83fe68d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:49.415200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfad51-4a94-49b9-a39d-0005f83fe68d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:49.451349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce0612-68ee-4bcd-af78-0e727bee4913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:49.452022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce0612-68ee-4bcd-af78-0e727bee4913 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.452310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce0612-68ee-4bcd-af78-0e727bee4913 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.452586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce0612-68ee-4bcd-af78-0e727bee4913 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.453296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce0612-68ee-4bcd-af78-0e727bee4913 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:49.453606 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce0612-68ee-4bcd-af78-0e727bee4913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:49.455158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce0612-68ee-4bcd-af78-0e727bee4913 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:49.460204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 450/906] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:49.466892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecba0f-94d8-41db-9b1a-82baa4e9662b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:49.526018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:49.575321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ecba0f-94d8-41db-9b1a-82baa4e9662b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:49.609992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17553c35-9cf3-4763-906c-70b95069f949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:49.610668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17553c35-9cf3-4763-906c-70b95069f949 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.610950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17553c35-9cf3-4763-906c-70b95069f949 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.611226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17553c35-9cf3-4763-906c-70b95069f949 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.611943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17553c35-9cf3-4763-906c-70b95069f949 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:49.612253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17553c35-9cf3-4763-906c-70b95069f949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:49.613618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17553c35-9cf3-4763-906c-70b95069f949 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:49.619008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 457/907] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:49.625469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce0612-68ee-4bcd-af78-0e727bee4913 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:49.662279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:49.701269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f706b-cd9a-4799-9c84-76c6cb4269ee None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.701592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f706b-cd9a-4799-9c84-76c6cb4269ee None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.701856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f706b-cd9a-4799-9c84-76c6cb4269ee None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.703746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 458/908] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:49.710325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ab297-94c1-433a-b711-fe7e710938eb None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.710931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ab297-94c1-433a-b711-fe7e710938eb None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.711436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ab297-94c1-433a-b711-fe7e710938eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.729901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e67ab297-94c1-433a-b711-fe7e710938eb None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:49.731599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e67ab297-94c1-433a-b711-fe7e710938eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:49.745438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce0612-68ee-4bcd-af78-0e727bee4913 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:49.788142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfea50-363d-43c5-99a3-3679bcb6f5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:49.789167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfea50-363d-43c5-99a3-3679bcb6f5ee None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.789658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfea50-363d-43c5-99a3-3679bcb6f5ee None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.790168 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dfea50-363d-43c5-99a3-3679bcb6f5ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.791310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfea50-363d-43c5-99a3-3679bcb6f5ee None admin] RBAC: Adding query filter params (interface=public, service_id=df01db0ef71e48afb825db7af0253953, region_id=RegionOne) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:49.791929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfea50-363d-43c5-99a3-3679bcb6f5ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:49.793586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dfea50-363d-43c5-99a3-3679bcb6f5ee None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:49.799913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 451/909] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 14 15:35:49 2022] GET /identity/v3/endpoints?service_id=df01db0ef71e48afb825db7af0253953&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:49.805128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfea50-363d-43c5-99a3-3679bcb6f5ee None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:49.811121 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 459/910] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:35:49.841461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ab297-94c1-433a-b711-fe7e710938eb None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:49.891360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:49.915649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:49.930954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dfea50-363d-43c5-99a3-3679bcb6f5ee None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:49.943075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ab297-94c1-433a-b711-fe7e710938eb None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:49.970907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b5ad6-e005-4306-bca3-a3d9e9551a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:49.972016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b5ad6-e005-4306-bca3-a3d9e9551a94 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.972560 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b5ad6-e005-4306-bca3-a3d9e9551a94 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.973040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b5ad6-e005-4306-bca3-a3d9e9551a94 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.974048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b5ad6-e005-4306-bca3-a3d9e9551a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:49.975955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b5ad6-e005-4306-bca3-a3d9e9551a94 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:49.983460 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 452/911] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 14 15:35:49 2022] GET /identity/v3/services => generated 1268 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:35:49.984705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fb857-a0b1-4661-874f-911d6d4cc0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:49.985382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fb857-a0b1-4661-874f-911d6d4cc0d7 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:49.985665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fb857-a0b1-4661-874f-911d6d4cc0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:49.985974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fb857-a0b1-4661-874f-911d6d4cc0d7 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:49.986728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fb857-a0b1-4661-874f-911d6d4cc0d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:49.988228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fb857-a0b1-4661-874f-911d6d4cc0d7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:50.068490 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 460/912] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:35:49 2022] POST /identity/v3/services => generated 241 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:35:50.627301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099531fe-4065-45dc-b15f-33679edff80f None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:50.628367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099531fe-4065-45dc-b15f-33679edff80f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:50.628872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099531fe-4065-45dc-b15f-33679edff80f None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:50.630777 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 453/913] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:50.636356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4a7b0-b0d6-45af-bc83-7f6e537b43de None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:50.636981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4a7b0-b0d6-45af-bc83-7f6e537b43de None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:50.637466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4a7b0-b0d6-45af-bc83-7f6e537b43de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:50.655494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-24d4a7b0-b0d6-45af-bc83-7f6e537b43de None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:50.657009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-24d4a7b0-b0d6-45af-bc83-7f6e537b43de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:50.664814 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844690b-324f-4bb0-bb57-ab6ef9899557 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:50.665350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844690b-324f-4bb0-bb57-ab6ef9899557 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:50.665629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844690b-324f-4bb0-bb57-ab6ef9899557 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:50.667363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 454/914] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:50.672397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caa868-c733-4e4a-a87a-866fe6ddd516 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:50.672916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caa868-c733-4e4a-a87a-866fe6ddd516 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:50.673252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caa868-c733-4e4a-a87a-866fe6ddd516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:50.691907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-49caa868-c733-4e4a-a87a-866fe6ddd516 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:50.693450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-49caa868-c733-4e4a-a87a-866fe6ddd516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:50.744473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 461/915] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:50.767246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4a7b0-b0d6-45af-bc83-7f6e537b43de None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:50.770367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 455/916] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:35:50.793662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caa868-c733-4e4a-a87a-866fe6ddd516 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:50.801634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:50.875184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4a7b0-b0d6-45af-bc83-7f6e537b43de None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:50.899017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:50.922621 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caa868-c733-4e4a-a87a-866fe6ddd516 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:50.922845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b5195-b45e-48aa-8587-30f2ff55a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:50.923820 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b5195-b45e-48aa-8587-30f2ff55a610 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:50.924403 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b5195-b45e-48aa-8587-30f2ff55a610 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:50.924868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b5195-b45e-48aa-8587-30f2ff55a610 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:50.941895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b5195-b45e-48aa-8587-30f2ff55a610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-650b5195-b45e-48aa-8587-30f2ff55a610 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:50.951149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:50.952289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:50.953314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 462/917] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 14 15:35:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:50.958704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b5195-b45e-48aa-8587-30f2ff55a610 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:50.964964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c937-0a66-4896-b5ce-331f9c32a5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:50.965980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c937-0a66-4896-b5ce-331f9c32a5df None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:50.966429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c937-0a66-4896-b5ce-331f9c32a5df None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:50.966866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077c937-0a66-4896-b5ce-331f9c32a5df None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:50.967682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077c937-0a66-4896-b5ce-331f9c32a5df None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:50.969207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077c937-0a66-4896-b5ce-331f9c32a5df None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-2077c937-0a66-4896-b5ce-331f9c32a5df None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:50.982150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:50.983137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 15:35:50.983975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:50.986905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 456/918] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:50.992906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c937-0a66-4896-b5ce-331f9c32a5df None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:51.092272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:51.100474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b5195-b45e-48aa-8587-30f2ff55a610 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:51.132762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077c937-0a66-4896-b5ce-331f9c32a5df None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:51.145149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e7121-b7e9-4620-889e-20e6861f6b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:51.146249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e7121-b7e9-4620-889e-20e6861f6b6a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:51.146671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e7121-b7e9-4620-889e-20e6861f6b6a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:51.147148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e7121-b7e9-4620-889e-20e6861f6b6a None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:51.148253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e7121-b7e9-4620-889e-20e6861f6b6a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:51.148759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e7121-b7e9-4620-889e-20e6861f6b6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:51.150440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e7121-b7e9-4620-889e-20e6861f6b6a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:51.160751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 463/919] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 14 15:35:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:51.171030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e7121-b7e9-4620-889e-20e6861f6b6a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:51.173343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a106362-faaa-4e8f-adb4-9efb0358cc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:51.174156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a106362-faaa-4e8f-adb4-9efb0358cc98 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:51.174969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a106362-faaa-4e8f-adb4-9efb0358cc98 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:51.175416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a106362-faaa-4e8f-adb4-9efb0358cc98 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:51.176375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a106362-faaa-4e8f-adb4-9efb0358cc98 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:51.176842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a106362-faaa-4e8f-adb4-9efb0358cc98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:51.178421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a106362-faaa-4e8f-adb4-9efb0358cc98 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:51.185847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 457/920] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:51.192792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a106362-faaa-4e8f-adb4-9efb0358cc98 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:51.210424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:51.294801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e7121-b7e9-4620-889e-20e6861f6b6a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:51.312854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a106362-faaa-4e8f-adb4-9efb0358cc98 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:51.346778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e848d6-644f-4063-bfd6-e0995c412f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:51.347854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e848d6-644f-4063-bfd6-e0995c412f16 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:51.348141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e848d6-644f-4063-bfd6-e0995c412f16 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:51.348418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e848d6-644f-4063-bfd6-e0995c412f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:51.352077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d323e-94a0-4755-a0fb-e2507a2d8cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:51.352973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d323e-94a0-4755-a0fb-e2507a2d8cca None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:51.353454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d323e-94a0-4755-a0fb-e2507a2d8cca None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:51.353960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d323e-94a0-4755-a0fb-e2507a2d8cca None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:51.354875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d323e-94a0-4755-a0fb-e2507a2d8cca None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:51.355434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d323e-94a0-4755-a0fb-e2507a2d8cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:51.357006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d323e-94a0-4755-a0fb-e2507a2d8cca None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:51.360565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e848d6-644f-4063-bfd6-e0995c412f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:51.362293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e848d6-644f-4063-bfd6-e0995c412f16 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:51.363327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 458/921] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 14 15:35:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:51.373855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d323e-94a0-4755-a0fb-e2507a2d8cca None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-c4e848d6-644f-4063-bfd6-e0995c412f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:51.382773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:51.383938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:51.385200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:51.385200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:51.385200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:51.385200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:51.388016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 464/922] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:51.393248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e848d6-644f-4063-bfd6-e0995c412f16 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:51.408734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:51.469339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:51.506708 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d323e-94a0-4755-a0fb-e2507a2d8cca None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:51.508752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e848d6-644f-4063-bfd6-e0995c412f16 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:51.550635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb45cf-98da-494f-9d39-795768662650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:51.551468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb45cf-98da-494f-9d39-795768662650 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:51.551873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb45cf-98da-494f-9d39-795768662650 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:51.552216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb45cf-98da-494f-9d39-795768662650 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:51.552663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00191726-25e8-4b4d-a2a7-3a31b48643e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:51.553082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb45cf-98da-494f-9d39-795768662650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:51.553459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb45cf-98da-494f-9d39-795768662650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:51.553974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00191726-25e8-4b4d-a2a7-3a31b48643e5 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:51.554451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00191726-25e8-4b4d-a2a7-3a31b48643e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:51.554918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00191726-25e8-4b4d-a2a7-3a31b48643e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:51.555124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb45cf-98da-494f-9d39-795768662650 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:51.556133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00191726-25e8-4b4d-a2a7-3a31b48643e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:51.557680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00191726-25e8-4b4d-a2a7-3a31b48643e5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:51.570304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 465/923] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:51.576079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb45cf-98da-494f-9d39-795768662650 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:51.640038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 459/924] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 14 15:35:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:51.649719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:51.690742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb45cf-98da-494f-9d39-795768662650 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:51.737739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6d6e3-5940-4366-b8d2-9d99cde5d020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:51.739138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6d6e3-5940-4366-b8d2-9d99cde5d020 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:51.739635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6d6e3-5940-4366-b8d2-9d99cde5d020 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:51.740161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6d6e3-5940-4366-b8d2-9d99cde5d020 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:51.749392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6d6e3-5940-4366-b8d2-9d99cde5d020 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:51.751242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6d6e3-5940-4366-b8d2-9d99cde5d020 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-cdf6d6e3-5940-4366-b8d2-9d99cde5d020 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:51.768307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:51.769357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:51.770365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:51.773113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 466/925] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:35:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:51.779548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00191726-25e8-4b4d-a2a7-3a31b48643e5 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:51.892980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00191726-25e8-4b4d-a2a7-3a31b48643e5 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:51.922389 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:51.929699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967f695-2183-4d56-8ef4-03cad6e50da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:51.930410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967f695-2183-4d56-8ef4-03cad6e50da8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:51.930698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967f695-2183-4d56-8ef4-03cad6e50da8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:51.930980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967f695-2183-4d56-8ef4-03cad6e50da8 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:51.931825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967f695-2183-4d56-8ef4-03cad6e50da8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:51.932133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967f695-2183-4d56-8ef4-03cad6e50da8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:51.934142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967f695-2183-4d56-8ef4-03cad6e50da8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:51.953187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 460/926] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 14 15:35:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:51.958941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6d6e3-5940-4366-b8d2-9d99cde5d020 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.068543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6d6e3-5940-4366-b8d2-9d99cde5d020 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.079972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f13f6-5fb6-427e-9ee9-d1ba425b7e90 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.085965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f13f6-5fb6-427e-9ee9-d1ba425b7e90 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.085965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f13f6-5fb6-427e-9ee9-d1ba425b7e90 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.085965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 461/927] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:52.088338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253964-1ce7-4cc5-9dc8-ffd42111e9a3 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.088624 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253964-1ce7-4cc5-9dc8-ffd42111e9a3 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.088897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253964-1ce7-4cc5-9dc8-ffd42111e9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.097480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:52.108466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22188df-6503-4fa8-be6c-c4db85dfc8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.109449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22188df-6503-4fa8-be6c-c4db85dfc8eb None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.109838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22188df-6503-4fa8-be6c-c4db85dfc8eb None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.110137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22188df-6503-4fa8-be6c-c4db85dfc8eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.112511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-55253964-1ce7-4cc5-9dc8-ffd42111e9a3 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:52.114272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-55253964-1ce7-4cc5-9dc8-ffd42111e9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:52.122408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22188df-6503-4fa8-be6c-c4db85dfc8eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.123961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22188df-6503-4fa8-be6c-c4db85dfc8eb None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-f22188df-6503-4fa8-be6c-c4db85dfc8eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:52.143289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:52.144249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:52.145145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:52.145145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:52.145145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:52.145145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:52.145145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 467/928] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:52.154158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22188df-6503-4fa8-be6c-c4db85dfc8eb None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.210048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 462/929] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:52.233395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253964-1ce7-4cc5-9dc8-ffd42111e9a3 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.275750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22188df-6503-4fa8-be6c-c4db85dfc8eb None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.312747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed385154-193d-482f-910d-493be2ea7795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.313411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed385154-193d-482f-910d-493be2ea7795 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.313696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed385154-193d-482f-910d-493be2ea7795 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.314002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed385154-193d-482f-910d-493be2ea7795 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.314729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed385154-193d-482f-910d-493be2ea7795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:52.315039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed385154-193d-482f-910d-493be2ea7795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.316652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed385154-193d-482f-910d-493be2ea7795 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.329076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 468/930] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:52.330341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41f80d-4ea8-4e0c-a767-451efe73917e None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.330630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41f80d-4ea8-4e0c-a767-451efe73917e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.330940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41f80d-4ea8-4e0c-a767-451efe73917e None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.332313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 469/931] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:52.334953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41f80d-4ea8-4e0c-a767-451efe73917e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.343065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253964-1ce7-4cc5-9dc8-ffd42111e9a3 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.363549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:52.375613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:52.382589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15dd05-cfee-4349-8e12-4abfcfa1dd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.383471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15dd05-cfee-4349-8e12-4abfcfa1dd13 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.383758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15dd05-cfee-4349-8e12-4abfcfa1dd13 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.384038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15dd05-cfee-4349-8e12-4abfcfa1dd13 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.384781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15dd05-cfee-4349-8e12-4abfcfa1dd13 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.386188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15dd05-cfee-4349-8e12-4abfcfa1dd13 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-ae15dd05-cfee-4349-8e12-4abfcfa1dd13 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:52.398767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:52.399785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:52.400674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 463/932] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Jul 14 15:35:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:52.401544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02d7d1-c73b-4382-92d2-ec51a7b8ef21 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.401544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02d7d1-c73b-4382-92d2-ec51a7b8ef21 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.402285 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02d7d1-c73b-4382-92d2-ec51a7b8ef21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.419535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5c02d7d1-c73b-4382-92d2-ec51a7b8ef21 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:52.420740 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5c02d7d1-c73b-4382-92d2-ec51a7b8ef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:52.450926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41f80d-4ea8-4e0c-a767-451efe73917e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.487283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7471d-9226-464c-a205-53e7dc7882ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.488009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7471d-9226-464c-a205-53e7dc7882ca None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.488295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7471d-9226-464c-a205-53e7dc7882ca None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.488573 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7471d-9226-464c-a205-53e7dc7882ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.497861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 464/933] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:52.497861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02d7d1-c73b-4382-92d2-ec51a7b8ef21 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.499143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7471d-9226-464c-a205-53e7dc7882ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.500817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7471d-9226-464c-a205-53e7dc7882ca None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-12f7471d-9226-464c-a205-53e7dc7882ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:52.516006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:52.516955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:52.517946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:52.517946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:52.517946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:52.517946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:52.517946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:52.517946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:52.517946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:52.517946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:52.520420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 470/934] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:52.521512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7471d-9226-464c-a205-53e7dc7882ca None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.617224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02d7d1-c73b-4382-92d2-ec51a7b8ef21 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.618567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:52.624823 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7471d-9226-464c-a205-53e7dc7882ca None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.632572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:52.655614 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877d404-0fd3-4976-9b66-7a04996a580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.656441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877d404-0fd3-4976-9b66-7a04996a580e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.656559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877d404-0fd3-4976-9b66-7a04996a580e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.656825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877d404-0fd3-4976-9b66-7a04996a580e None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.657540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877d404-0fd3-4976-9b66-7a04996a580e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:52.657879 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877d404-0fd3-4976-9b66-7a04996a580e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.658985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe572b-99f0-44be-8bf9-851034577ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.659269 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877d404-0fd3-4976-9b66-7a04996a580e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.659795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe572b-99f0-44be-8bf9-851034577ec2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.660081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe572b-99f0-44be-8bf9-851034577ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.660363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe572b-99f0-44be-8bf9-851034577ec2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.661066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe572b-99f0-44be-8bf9-851034577ec2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.662452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe572b-99f0-44be-8bf9-851034577ec2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.665100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 465/935] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:52.666232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877d404-0fd3-4976-9b66-7a04996a580e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-5bbe572b-99f0-44be-8bf9-851034577ec2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:52.678778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:52.679855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:52.680718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 471/936] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:52.681761 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe572b-99f0-44be-8bf9-851034577ec2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.777665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877d404-0fd3-4976-9b66-7a04996a580e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.789925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe572b-99f0-44be-8bf9-851034577ec2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.812745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667684d0-120c-48d8-ad58-28c3e7434a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.813502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667684d0-120c-48d8-ad58-28c3e7434a1e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.813893 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667684d0-120c-48d8-ad58-28c3e7434a1e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.814178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667684d0-120c-48d8-ad58-28c3e7434a1e None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.814971 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667684d0-120c-48d8-ad58-28c3e7434a1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:52.815550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667684d0-120c-48d8-ad58-28c3e7434a1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.816977 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667684d0-120c-48d8-ad58-28c3e7434a1e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.826568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcc3be-a7b3-4352-a945-cb9cb1e9c0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.827233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc3be-a7b3-4352-a945-cb9cb1e9c0d0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.827522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc3be-a7b3-4352-a945-cb9cb1e9c0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.827797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcc3be-a7b3-4352-a945-cb9cb1e9c0d0 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.828510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcc3be-a7b3-4352-a945-cb9cb1e9c0d0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:52.828821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcc3be-a7b3-4352-a945-cb9cb1e9c0d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.835020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 466/937] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:52.835199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcc3be-a7b3-4352-a945-cb9cb1e9c0d0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.835295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667684d0-120c-48d8-ad58-28c3e7434a1e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.841583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 472/938] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:52.842389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcc3be-a7b3-4352-a945-cb9cb1e9c0d0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:52.871887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:52.881160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:52.948708 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667684d0-120c-48d8-ad58-28c3e7434a1e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.949482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcc3be-a7b3-4352-a945-cb9cb1e9c0d0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:52.985320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b81347-f3dd-4084-8ae4-32cb7c967cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.986038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b81347-f3dd-4084-8ae4-32cb7c967cd7 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.986273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9549b9-26b4-4a68-b0e9-254a0e8a8d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:52.986409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b81347-f3dd-4084-8ae4-32cb7c967cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.986591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b81347-f3dd-4084-8ae4-32cb7c967cd7 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.987164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9549b9-26b4-4a68-b0e9-254a0e8a8d43 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:52.987326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b81347-f3dd-4084-8ae4-32cb7c967cd7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:52.987433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9549b9-26b4-4a68-b0e9-254a0e8a8d43 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:52.987591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b81347-f3dd-4084-8ae4-32cb7c967cd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.987767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9549b9-26b4-4a68-b0e9-254a0e8a8d43 None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/613c0627a67041cd858b61ecc3d08421/roles/5ccecebd4103442484b3dadfbfbf47c4` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:52.988969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b81347-f3dd-4084-8ae4-32cb7c967cd7 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.990128 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9549b9-26b4-4a68-b0e9-254a0e8a8d43 None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=613c0627a67041cd858b61ecc3d08421, role_id=5ccecebd4103442484b3dadfbfbf47c4)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:52.991830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9549b9-26b4-4a68-b0e9-254a0e8a8d43 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:52.999228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 467/939] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:35:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:53.000185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b81347-f3dd-4084-8ae4-32cb7c967cd7 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:53.086129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 473/940] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:35:52 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/613c0627a67041cd858b61ecc3d08421/roles/5ccecebd4103442484b3dadfbfbf47c4 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:35:53.087265 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9549b9-26b4-4a68-b0e9-254a0e8a8d43 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:53.114698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:53.118167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b81347-f3dd-4084-8ae4-32cb7c967cd7 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:53.163231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f07fd-17ba-44e8-9ee7-4e29f98eaf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:53.164319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f07fd-17ba-44e8-9ee7-4e29f98eaf23 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:53.164909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f07fd-17ba-44e8-9ee7-4e29f98eaf23 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:53.165384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f07fd-17ba-44e8-9ee7-4e29f98eaf23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:53.166538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f07fd-17ba-44e8-9ee7-4e29f98eaf23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:53.168331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f07fd-17ba-44e8-9ee7-4e29f98eaf23 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:53.208512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:53.226774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9549b9-26b4-4a68-b0e9-254a0e8a8d43 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:53.255759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 468/941] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 14 15:35:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:53.273037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fa168-3609-41ca-8b74-9c56e78f93dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:53.273853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fa168-3609-41ca-8b74-9c56e78f93dc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:53.274144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fa168-3609-41ca-8b74-9c56e78f93dc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:53.274438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fa168-3609-41ca-8b74-9c56e78f93dc None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:53.275266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807fa168-3609-41ca-8b74-9c56e78f93dc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:53.275653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807fa168-3609-41ca-8b74-9c56e78f93dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:53.277108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807fa168-3609-41ca-8b74-9c56e78f93dc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:53.283883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 474/942] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:35:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:53.289987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f07fd-17ba-44e8-9ee7-4e29f98eaf23 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:53.410971 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f07fd-17ba-44e8-9ee7-4e29f98eaf23 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:53.452519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75090175-3ee1-4450-be13-3f4065815157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:53.453195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75090175-3ee1-4450-be13-3f4065815157 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:53.453481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75090175-3ee1-4450-be13-3f4065815157 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:53.453760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75090175-3ee1-4450-be13-3f4065815157 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:53.454588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75090175-3ee1-4450-be13-3f4065815157 None admin] RBAC: Adding query filter params (interface=public, service_id=5a9d453e0f0746fbb94458d940d04ceb, region_id=RegionOne) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:53.454906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75090175-3ee1-4450-be13-3f4065815157 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:53.456292 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75090175-3ee1-4450-be13-3f4065815157 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:53.462075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 469/943] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 14 15:35:53 2022] GET /identity/v3/endpoints?service_id=5a9d453e0f0746fbb94458d940d04ceb&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:53.468673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fa168-3609-41ca-8b74-9c56e78f93dc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:53.580046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fa168-3609-41ca-8b74-9c56e78f93dc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:53.615188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2119d6e-8ed8-4495-95da-19f658889875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:53.616274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2119d6e-8ed8-4495-95da-19f658889875 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:53.616740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2119d6e-8ed8-4495-95da-19f658889875 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:53.617197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2119d6e-8ed8-4495-95da-19f658889875 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:53.618079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2119d6e-8ed8-4495-95da-19f658889875 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:53.619634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2119d6e-8ed8-4495-95da-19f658889875 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:53.626795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 475/944] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 14 15:35:53 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:35:53.797376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e542cca-eced-4a2b-8186-38685ce142ef None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:53.797903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e542cca-eced-4a2b-8186-38685ce142ef None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:53.797985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e542cca-eced-4a2b-8186-38685ce142ef None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:53.799674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 470/945] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:53.805094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38467946-9eaf-4633-9617-ce478e682fd2 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:53.805387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38467946-9eaf-4633-9617-ce478e682fd2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:53.805674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38467946-9eaf-4633-9617-ce478e682fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:53.823097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-38467946-9eaf-4633-9617-ce478e682fd2 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:53.824847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-38467946-9eaf-4633-9617-ce478e682fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:53.827271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:53.901931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 476/946] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:53.919555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e542cca-eced-4a2b-8186-38685ce142ef None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:53.944479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:54.029206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e542cca-eced-4a2b-8186-38685ce142ef None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:54.064896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:54.065752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:54.066109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:54.066385 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:54.067104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:54.068538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:54.069959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WARNING keystone.server.flask.common [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 15:35:54.120098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.sql.core [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 15:35:54.120098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 15:35:54.120098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [parameters: {'id': '08ef3db4be5740d5883022fa36826000', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 15:35:54.120098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92133) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 08ef3db4be5740d5883022fa36826000.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 08ef3db4be5740d5883022fa36826000. Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:35:54.138715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application result.read() Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 15:35:54.139611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application yield resource Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise self.value Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application session.commit() Jul 14 15:35:54.141115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self.session.flush() Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application compat.raise_( Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 15:35:54.143277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 15:35:54.145415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application result.read() Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:35:54.147332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [parameters: {'id': '08ef3db4be5740d5883022fa36826000', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:54.149580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:54.150640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 08ef3db4be5740d5883022fa36826000. Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:54.151823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 471/947] 10.222.0.47 () {60 vars in 1140 bytes} [Thu Jul 14 15:35:53 2022] POST /identity/v3/projects => generated 262 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:54.153087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38467946-9eaf-4633-9617-ce478e682fd2 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:54.257344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38467946-9eaf-4633-9617-ce478e682fd2 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:54.295699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761cb08-bdf3-4198-ac9d-8492e8d56f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:54.296506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761cb08-bdf3-4198-ac9d-8492e8d56f9e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:54.296788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761cb08-bdf3-4198-ac9d-8492e8d56f9e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:54.297065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761cb08-bdf3-4198-ac9d-8492e8d56f9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:54.304389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761cb08-bdf3-4198-ac9d-8492e8d56f9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:54.305856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761cb08-bdf3-4198-ac9d-8492e8d56f9e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:54.308145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-4761cb08-bdf3-4198-ac9d-8492e8d56f9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:54.323638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:54.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:54.328414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:54.328414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:54.328414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:54.328414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:54.328414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:54.328414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:54.328414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:54.328414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:54.328414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 477/948] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:54.333848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:54.421619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:54.446438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec085a0b-a014-4690-8c23-c7d611ea6b76 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:54.482616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67de86-a122-401d-abd2-e444f0059c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:54.483288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67de86-a122-401d-abd2-e444f0059c7a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:54.483575 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67de86-a122-401d-abd2-e444f0059c7a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:54.483855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67de86-a122-401d-abd2-e444f0059c7a None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:54.484623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67de86-a122-401d-abd2-e444f0059c7a None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:54.484932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67de86-a122-401d-abd2-e444f0059c7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:54.486620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67de86-a122-401d-abd2-e444f0059c7a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:54.508887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 472/949] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:35:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:55.338169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beeff7-8205-4e70-96ed-2666b1ba6119 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.338661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beeff7-8205-4e70-96ed-2666b1ba6119 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.338661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beeff7-8205-4e70-96ed-2666b1ba6119 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.340333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 478/950] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:55.345622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe35bbe-dcca-4e9d-b4a4-8a1e5ad891a2 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.346041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe35bbe-dcca-4e9d-b4a4-8a1e5ad891a2 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.346230 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe35bbe-dcca-4e9d-b4a4-8a1e5ad891a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.365382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-abe35bbe-dcca-4e9d-b4a4-8a1e5ad891a2 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:55.366940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-abe35bbe-dcca-4e9d-b4a4-8a1e5ad891a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:55.439009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 473/951] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:55.463427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beeff7-8205-4e70-96ed-2666b1ba6119 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:55.509586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0e4f-7e3b-4242-9ebd-04874555a244 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.509896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0e4f-7e3b-4242-9ebd-04874555a244 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.510176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a0e4f-7e3b-4242-9ebd-04874555a244 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.512047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 474/952] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:35:55.516725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0865d2-85f4-4b1d-b7c5-b4dd4c02b42a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.517004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0865d2-85f4-4b1d-b7c5-b4dd4c02b42a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.517293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0865d2-85f4-4b1d-b7c5-b4dd4c02b42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.535997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-cf0865d2-85f4-4b1d-b7c5-b4dd4c02b42a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:55.537201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-cf0865d2-85f4-4b1d-b7c5-b4dd4c02b42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:55.586822 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beeff7-8205-4e70-96ed-2666b1ba6119 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:55.609771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:55.614051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 475/953] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:35:55.624684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206a333-c8b4-4842-8173-1c5b45e8c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:55.625794 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206a333-c8b4-4842-8173-1c5b45e8c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.626329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206a333-c8b4-4842-8173-1c5b45e8c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.626850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206a333-c8b4-4842-8173-1c5b45e8c7f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.630941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0865d2-85f4-4b1d-b7c5-b4dd4c02b42a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:55.639200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206a333-c8b4-4842-8173-1c5b45e8c7f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-c206a333-c8b4-4842-8173-1c5b45e8c7f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:35:55.647678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:55.648647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:35:55.649523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:55.652818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 479/954] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 14 15:35:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:55.658980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206a333-c8b4-4842-8173-1c5b45e8c7f3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:55.725885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:55.739607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0865d2-85f4-4b1d-b7c5-b4dd4c02b42a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:55.775345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206a333-c8b4-4842-8173-1c5b45e8c7f3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:55.776316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4e03b-fccc-4e37-a547-0817b70f1ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:55.777153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4e03b-fccc-4e37-a547-0817b70f1ed0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.777448 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4e03b-fccc-4e37-a547-0817b70f1ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.777735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4e03b-fccc-4e37-a547-0817b70f1ed0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.778433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4e03b-fccc-4e37-a547-0817b70f1ed0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:55.779817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4e03b-fccc-4e37-a547-0817b70f1ed0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-10b4e03b-fccc-4e37-a547-0817b70f1ed0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:55.791832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:55.792925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:55.794415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 476/955] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:55.795297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1e1e1-4255-4043-8c41-80f46a6e1199 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.795297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1e1e1-4255-4043-8c41-80f46a6e1199 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.795297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1e1e1-4255-4043-8c41-80f46a6e1199 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.796569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 477/956] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:55.801144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1e1e1-4255-4043-8c41-80f46a6e1199 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:55.817222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b3bed-95af-4357-9c3b-293499ef27ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:55.817926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b3bed-95af-4357-9c3b-293499ef27ed None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.818206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b3bed-95af-4357-9c3b-293499ef27ed None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.818476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b3bed-95af-4357-9c3b-293499ef27ed None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.819214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b3bed-95af-4357-9c3b-293499ef27ed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:55.819516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b3bed-95af-4357-9c3b-293499ef27ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:55.820867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b3bed-95af-4357-9c3b-293499ef27ed None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:55.829753 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 480/957] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 14 15:35:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:55.831343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41524685-c9d4-48c6-9b6e-aeda1bb0112c None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.831636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41524685-c9d4-48c6-9b6e-aeda1bb0112c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.831917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41524685-c9d4-48c6-9b6e-aeda1bb0112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.848546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-41524685-c9d4-48c6-9b6e-aeda1bb0112c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:55.849728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-41524685-c9d4-48c6-9b6e-aeda1bb0112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:55.873692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:55.917836 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1e1e1-4255-4043-8c41-80f46a6e1199 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:55.922008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 481/958] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:55.922008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41524685-c9d4-48c6-9b6e-aeda1bb0112c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:55.958803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba71f4-48e3-48d9-b73f-242fb4ce6bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:55.959484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba71f4-48e3-48d9-b73f-242fb4ce6bc0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:55.959767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba71f4-48e3-48d9-b73f-242fb4ce6bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:55.960044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba71f4-48e3-48d9-b73f-242fb4ce6bc0 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:55.960754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba71f4-48e3-48d9-b73f-242fb4ce6bc0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:55.961055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba71f4-48e3-48d9-b73f-242fb4ce6bc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:55.962669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba71f4-48e3-48d9-b73f-242fb4ce6bc0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:55.972595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 478/959] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:35:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:55.972595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba71f4-48e3-48d9-b73f-242fb4ce6bc0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:55.983144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:56.037833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41524685-c9d4-48c6-9b6e-aeda1bb0112c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.074735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1a8f3-0f42-4c08-ab29-deba9a4d157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.075534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1a8f3-0f42-4c08-ab29-deba9a4d157b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.075812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1a8f3-0f42-4c08-ab29-deba9a4d157b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.076076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1a8f3-0f42-4c08-ab29-deba9a4d157b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.087570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1a8f3-0f42-4c08-ab29-deba9a4d157b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:56.088950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1a8f3-0f42-4c08-ab29-deba9a4d157b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:56.089394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba71f4-48e3-48d9-b73f-242fb4ce6bc0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-9cf1a8f3-0f42-4c08-ab29-deba9a4d157b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:56.107809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:56.111196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:56.112300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:56.112300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:56.112300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:56.112300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:56.112300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 482/960] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:56.112300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1a8f3-0f42-4c08-ab29-deba9a4d157b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:56.118135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:56.128675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b08d34-8485-41f6-a67c-ebed43deda61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.129450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b08d34-8485-41f6-a67c-ebed43deda61 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.129693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b08d34-8485-41f6-a67c-ebed43deda61 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.130010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b08d34-8485-41f6-a67c-ebed43deda61 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.134225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b08d34-8485-41f6-a67c-ebed43deda61 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:56.134225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b08d34-8485-41f6-a67c-ebed43deda61 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-36b08d34-8485-41f6-a67c-ebed43deda61 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:56.145317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:56.146546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:56.147433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 479/961] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:56.148315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b08d34-8485-41f6-a67c-ebed43deda61 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:56.224089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1a8f3-0f42-4c08-ab29-deba9a4d157b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.225643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:56.252400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b08d34-8485-41f6-a67c-ebed43deda61 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.259643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904ffda-2bdb-4a7e-89a1-5bc402a6e252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.260405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904ffda-2bdb-4a7e-89a1-5bc402a6e252 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.260761 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904ffda-2bdb-4a7e-89a1-5bc402a6e252 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.261110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904ffda-2bdb-4a7e-89a1-5bc402a6e252 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.262457 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904ffda-2bdb-4a7e-89a1-5bc402a6e252 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:56.262865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904ffda-2bdb-4a7e-89a1-5bc402a6e252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:56.264305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a904ffda-2bdb-4a7e-89a1-5bc402a6e252 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:56.269109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 483/962] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:35:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:56.270024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904ffda-2bdb-4a7e-89a1-5bc402a6e252 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:56.307848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60b6c3-1e23-4509-9bfc-f0264010a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.309760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60b6c3-1e23-4509-9bfc-f0264010a14f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.310281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60b6c3-1e23-4509-9bfc-f0264010a14f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.310734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60b6c3-1e23-4509-9bfc-f0264010a14f None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.311636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60b6c3-1e23-4509-9bfc-f0264010a14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:56.312101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60b6c3-1e23-4509-9bfc-f0264010a14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:56.313653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60b6c3-1e23-4509-9bfc-f0264010a14f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:56.332711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 480/963] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:56.334102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60b6c3-1e23-4509-9bfc-f0264010a14f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:56.385179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904ffda-2bdb-4a7e-89a1-5bc402a6e252 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.403455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:56.425893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92911829-a9ea-4906-ad94-9ffb4659784a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.426762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92911829-a9ea-4906-ad94-9ffb4659784a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.427170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92911829-a9ea-4906-ad94-9ffb4659784a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.427548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92911829-a9ea-4906-ad94-9ffb4659784a None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.428378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92911829-a9ea-4906-ad94-9ffb4659784a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:56.428760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92911829-a9ea-4906-ad94-9ffb4659784a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:56.430567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92911829-a9ea-4906-ad94-9ffb4659784a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:56.442892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 484/964] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:35:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:56.443947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92911829-a9ea-4906-ad94-9ffb4659784a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:56.471447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60b6c3-1e23-4509-9bfc-f0264010a14f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.483258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:56.521297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c0dfe-597e-4d38-b03c-a3723e0bf341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.522501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c0dfe-597e-4d38-b03c-a3723e0bf341 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.522783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c0dfe-597e-4d38-b03c-a3723e0bf341 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.523066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c0dfe-597e-4d38-b03c-a3723e0bf341 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.532959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c0dfe-597e-4d38-b03c-a3723e0bf341 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:56.535039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c0dfe-597e-4d38-b03c-a3723e0bf341 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-400c0dfe-597e-4d38-b03c-a3723e0bf341 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:56.553198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:56.554276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 15:35:56.555237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:56.561974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 481/965] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:35:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:56.564418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c0dfe-597e-4d38-b03c-a3723e0bf341 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:56.564634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92911829-a9ea-4906-ad94-9ffb4659784a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.614724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b61541-f492-461e-9391-416080b83d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.615408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b61541-f492-461e-9391-416080b83d1a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.615777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b61541-f492-461e-9391-416080b83d1a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.616420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b61541-f492-461e-9391-416080b83d1a None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.617474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b61541-f492-461e-9391-416080b83d1a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:56.617793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b61541-f492-461e-9391-416080b83d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:56.619359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b61541-f492-461e-9391-416080b83d1a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:56.629884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 485/966] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:35:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:56.633208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b61541-f492-461e-9391-416080b83d1a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:56.685288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c0dfe-597e-4d38-b03c-a3723e0bf341 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.695544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:56.724877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f8950-5dea-4519-a752-42de92ded727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.725569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f8950-5dea-4519-a752-42de92ded727 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.725885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f8950-5dea-4519-a752-42de92ded727 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.726175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f8950-5dea-4519-a752-42de92ded727 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.727028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f8950-5dea-4519-a752-42de92ded727 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:56.727341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f8950-5dea-4519-a752-42de92ded727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:56.728790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f8950-5dea-4519-a752-42de92ded727 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:56.742412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:56.753357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 482/967] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 14 15:35:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:56.753541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b61541-f492-461e-9391-416080b83d1a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.753636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fb62b-c079-45fa-8ad3-208ba2ad9760 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.753636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fb62b-c079-45fa-8ad3-208ba2ad9760 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.753636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745fb62b-c079-45fa-8ad3-208ba2ad9760 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.753636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 483/968] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:56.757427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fb62b-c079-45fa-8ad3-208ba2ad9760 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:56.790994 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249c118-a17d-465b-941d-0c270877e28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.791658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249c118-a17d-465b-941d-0c270877e28c None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.791941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249c118-a17d-465b-941d-0c270877e28c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.792212 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249c118-a17d-465b-941d-0c270877e28c None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.793006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249c118-a17d-465b-941d-0c270877e28c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:56.794393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249c118-a17d-465b-941d-0c270877e28c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:56.867439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745fb62b-c079-45fa-8ad3-208ba2ad9760 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:56.898951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 486/969] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 14 15:35:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:56.898951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e4f5-7039-44a4-a3f3-123a8397601d None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.898951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e4f5-7039-44a4-a3f3-123a8397601d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.898951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e4f5-7039-44a4-a3f3-123a8397601d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.917126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7832e4f5-7039-44a4-a3f3-123a8397601d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:56.918618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7832e4f5-7039-44a4-a3f3-123a8397601d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:56.925055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cf0fb-c445-40b1-8b33-975a31243754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:56.925742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cf0fb-c445-40b1-8b33-975a31243754 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:56.926045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cf0fb-c445-40b1-8b33-975a31243754 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:56.926320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cf0fb-c445-40b1-8b33-975a31243754 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:56.966954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:57.010352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cf0fb-c445-40b1-8b33-975a31243754 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:57.012545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782cf0fb-c445-40b1-8b33-975a31243754 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:57.017989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 487/970] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:57.046043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e4f5-7039-44a4-a3f3-123a8397601d None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:57.147668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 484/971] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:35:56 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:35:57.156580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cf0fb-c445-40b1-8b33-975a31243754 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:57.163692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:57.184717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e4f5-7039-44a4-a3f3-123a8397601d None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:57.223089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:57.223823 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:57.224200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:57.224591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:57.225390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:57.225930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:57.227966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:57.229338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING keystone.server.flask.common [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 15:35:57.271367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782cf0fb-c445-40b1-8b33-975a31243754 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:57.279710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.sql.core [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 15:35:57.279710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 15:35:57.279710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [parameters: {'user_id': '531c64d218224903a739c5d850ab9df4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 15:35:57.279710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92134) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application result.read() Jul 14 15:35:57.287440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:35:57.288568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application yield resource Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application next(self.gen) Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise self.value Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application session.commit() Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 15:35:57.289751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.session.flush() Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application compat.raise_( Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 15:35:57.290948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 15:35:57.292201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application conn.query(q) Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application result.read() Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 15:35:57.293237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [parameters: {'user_id': '531c64d218224903a739c5d850ab9df4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:57.294378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 15:35:57.295365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 15:35:57.297874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 15:35:57.297874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 15:35:57.297874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 15:35:57.297874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 15:35:57.297874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 15:35:57.297874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:57.297874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 488/972] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 14 15:35:57 2022] POST /identity/v3/users => generated 160 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:57.302850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:57.333222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db891ef-2fb4-4321-97a4-1d15d340046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:57.334551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db891ef-2fb4-4321-97a4-1d15d340046d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:57.335038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db891ef-2fb4-4321-97a4-1d15d340046d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:57.335708 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db891ef-2fb4-4321-97a4-1d15d340046d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:57.350254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db891ef-2fb4-4321-97a4-1d15d340046d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:57.353724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db891ef-2fb4-4321-97a4-1d15d340046d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-1db891ef-2fb4-4321-97a4-1d15d340046d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:35:57.374175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:57.375316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:57.376439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:35:57.376439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:35:57.376439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:35:57.376439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:57.382240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 485/973] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:57.384691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db891ef-2fb4-4321-97a4-1d15d340046d None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:57.447045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f326e8-821e-4e19-b035-a8ce4db9bdd1 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:57.483409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35821aa-af1b-4e00-9544-c777ecfb8832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:57.484135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35821aa-af1b-4e00-9544-c777ecfb8832 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:57.484421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35821aa-af1b-4e00-9544-c777ecfb8832 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:57.484697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35821aa-af1b-4e00-9544-c777ecfb8832 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:57.492783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35821aa-af1b-4e00-9544-c777ecfb8832 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:57.494447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35821aa-af1b-4e00-9544-c777ecfb8832 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-c35821aa-af1b-4e00-9544-c777ecfb8832 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:35:57.513900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:57.515106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 15:35:57.516237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:35:57.516930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db891ef-2fb4-4321-97a4-1d15d340046d None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:57.520018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 489/974] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:35:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:35:57.525865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35821aa-af1b-4e00-9544-c777ecfb8832 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:57.529352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:57.533085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:57.554967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c15f6a-f1a9-4244-b507-9ef9b88b1f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:57.555667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c15f6a-f1a9-4244-b507-9ef9b88b1f37 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:57.555953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c15f6a-f1a9-4244-b507-9ef9b88b1f37 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:57.556450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c15f6a-f1a9-4244-b507-9ef9b88b1f37 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:57.556960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c15f6a-f1a9-4244-b507-9ef9b88b1f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:57.557272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c15f6a-f1a9-4244-b507-9ef9b88b1f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:57.559277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c15f6a-f1a9-4244-b507-9ef9b88b1f37 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:57.575802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 486/975] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:57.577121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c15f6a-f1a9-4244-b507-9ef9b88b1f37 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:57.643167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35821aa-af1b-4e00-9544-c777ecfb8832 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:57.681263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c15f6a-f1a9-4244-b507-9ef9b88b1f37 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:57.682388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c03ff-b203-4e50-87e5-06d0ca02da17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:57.684211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c03ff-b203-4e50-87e5-06d0ca02da17 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:57.684724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c03ff-b203-4e50-87e5-06d0ca02da17 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:57.685223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c03ff-b203-4e50-87e5-06d0ca02da17 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:57.686287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c03ff-b203-4e50-87e5-06d0ca02da17 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:57.686808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c03ff-b203-4e50-87e5-06d0ca02da17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:57.688465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c03ff-b203-4e50-87e5-06d0ca02da17 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:57.718033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db27c6-22c4-475a-88cc-104c3ae43b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:57.718841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db27c6-22c4-475a-88cc-104c3ae43b7f None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:57.719135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db27c6-22c4-475a-88cc-104c3ae43b7f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:57.719413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db27c6-22c4-475a-88cc-104c3ae43b7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:57.726635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db27c6-22c4-475a-88cc-104c3ae43b7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:57.728092 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db27c6-22c4-475a-88cc-104c3ae43b7f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:57.733340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 490/976] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 14 15:35:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-23db27c6-22c4-475a-88cc-104c3ae43b7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:35:57.751031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:57.752004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:57.752925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:35:57.752925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:35:57.752925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:35:57.752925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:35:57.752925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:35:57.752925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:35:57.752925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:35:57.752925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:57.756714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 487/977] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:35:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:57.763653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c03ff-b203-4e50-87e5-06d0ca02da17 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:57.833570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:57.882225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c03ff-b203-4e50-87e5-06d0ca02da17 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:57.921284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9917c-b35e-40a0-a273-aab56610e604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:57.921988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9917c-b35e-40a0-a273-aab56610e604 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:57.922273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9917c-b35e-40a0-a273-aab56610e604 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:57.922551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9917c-b35e-40a0-a273-aab56610e604 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:57.923344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9917c-b35e-40a0-a273-aab56610e604 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:57.923650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9917c-b35e-40a0-a273-aab56610e604 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:57.925075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9917c-b35e-40a0-a273-aab56610e604 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:57.938448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 491/978] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:35:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:57.944962 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db27c6-22c4-475a-88cc-104c3ae43b7f None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:57.983009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:58.063189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db27c6-22c4-475a-88cc-104c3ae43b7f None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:58.101535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68a101-7800-46be-9080-c330aa0bf0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:58.102242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68a101-7800-46be-9080-c330aa0bf0b5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:58.102528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68a101-7800-46be-9080-c330aa0bf0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:58.102806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68a101-7800-46be-9080-c330aa0bf0b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:58.103650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68a101-7800-46be-9080-c330aa0bf0b5 None admin] RBAC: Adding query filter params (role.id=5ccecebd4103442484b3dadfbfbf47c4, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=613c0627a67041cd858b61ecc3d08421) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:58.103961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68a101-7800-46be-9080-c330aa0bf0b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:58.105450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68a101-7800-46be-9080-c330aa0bf0b5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:58.118085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 488/979] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:35:57 2022] GET /identity/v3/role_assignments?user.id=613c0627a67041cd858b61ecc3d08421&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=5ccecebd4103442484b3dadfbfbf47c4 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:35:58.368481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb2cb-5ba5-41b1-94a5-b7f983ef89fc None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:58.368834 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb2cb-5ba5-41b1-94a5-b7f983ef89fc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:58.369018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb2cb-5ba5-41b1-94a5-b7f983ef89fc None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:58.370531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 492/980] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:58.375997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743690aa-64b6-4e7a-b333-5a9ffad96c07 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:58.376299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743690aa-64b6-4e7a-b333-5a9ffad96c07 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:58.376579 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743690aa-64b6-4e7a-b333-5a9ffad96c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:58.393190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-743690aa-64b6-4e7a-b333-5a9ffad96c07 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:58.394411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-743690aa-64b6-4e7a-b333-5a9ffad96c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:58.464246 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 489/981] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:58.486671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bb2cb-5ba5-41b1-94a5-b7f983ef89fc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:58.589158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bb2cb-5ba5-41b1-94a5-b7f983ef89fc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:58.609602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:58.624361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e64e6f-5ce0-4c60-b6e4-ff954ca7cabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:58.624990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e64e6f-5ce0-4c60-b6e4-ff954ca7cabf None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:58.625263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e64e6f-5ce0-4c60-b6e4-ff954ca7cabf None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:58.625535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e64e6f-5ce0-4c60-b6e4-ff954ca7cabf None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:58.626301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e64e6f-5ce0-4c60-b6e4-ff954ca7cabf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:58.627662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e64e6f-5ce0-4c60-b6e4-ff954ca7cabf None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:58.703779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 493/982] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:35:58 2022] POST /identity/v3/services => generated 242 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:35:59.018320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3a8b7-ab22-4ae0-86bf-d14cfa38dac9 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:59.018835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3a8b7-ab22-4ae0-86bf-d14cfa38dac9 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:59.018955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3a8b7-ab22-4ae0-86bf-d14cfa38dac9 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:59.020462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 490/983] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:35:59.025518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee2492-5376-4dfe-a159-f74dea9f79ba None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:59.031778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee2492-5376-4dfe-a159-f74dea9f79ba None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:59.031778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee2492-5376-4dfe-a159-f74dea9f79ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:59.047387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-84ee2492-5376-4dfe-a159-f74dea9f79ba None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:35:59.048901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-84ee2492-5376-4dfe-a159-f74dea9f79ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:35:59.126597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 494/984] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:35:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:35:59.148740 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3a8b7-ab22-4ae0-86bf-d14cfa38dac9 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:59.178029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:59.257671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3a8b7-ab22-4ae0-86bf-d14cfa38dac9 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:59.293568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcec066-f663-4294-9b0b-9978800542ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:59.294314 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcec066-f663-4294-9b0b-9978800542ea None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:59.294592 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcec066-f663-4294-9b0b-9978800542ea None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:59.294860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcec066-f663-4294-9b0b-9978800542ea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:59.295507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcec066-f663-4294-9b0b-9978800542ea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:59.296865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcec066-f663-4294-9b0b-9978800542ea None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-3fcec066-f663-4294-9b0b-9978800542ea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:35:59.309513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:35:59.310766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:35:59.311623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:35:59.316139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 491/985] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:35:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:35:59.316957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee2492-5376-4dfe-a159-f74dea9f79ba None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:59.419679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee2492-5376-4dfe-a159-f74dea9f79ba None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:59.426977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:59.452963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db28040-2336-404d-bef0-ba905ed59a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:59.453632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db28040-2336-404d-bef0-ba905ed59a70 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:59.453951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db28040-2336-404d-bef0-ba905ed59a70 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:59.454222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db28040-2336-404d-bef0-ba905ed59a70 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:59.454927 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db28040-2336-404d-bef0-ba905ed59a70 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:59.455229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db28040-2336-404d-bef0-ba905ed59a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:59.456579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db28040-2336-404d-bef0-ba905ed59a70 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:59.461486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 495/986] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:35:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:59.467494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcec066-f663-4294-9b0b-9978800542ea None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:59.559116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:35:59.568173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcec066-f663-4294-9b0b-9978800542ea None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:59.602221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f5765-12ec-4a50-97a7-238a22bdd761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:59.602884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f5765-12ec-4a50-97a7-238a22bdd761 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:59.603162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f5765-12ec-4a50-97a7-238a22bdd761 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:59.603431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f5765-12ec-4a50-97a7-238a22bdd761 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:59.604132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f5765-12ec-4a50-97a7-238a22bdd761 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:59.604436 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f5765-12ec-4a50-97a7-238a22bdd761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:59.606362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f5765-12ec-4a50-97a7-238a22bdd761 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:59.616023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 492/987] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:35:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:59.617554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db28040-2336-404d-bef0-ba905ed59a70 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:59.717172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db28040-2336-404d-bef0-ba905ed59a70 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:59.751907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c8d1a-3aff-4c78-a82d-f4454df5a675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:59.752876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c8d1a-3aff-4c78-a82d-f4454df5a675 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:59.753350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c8d1a-3aff-4c78-a82d-f4454df5a675 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:59.753835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c8d1a-3aff-4c78-a82d-f4454df5a675 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:59.754809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c8d1a-3aff-4c78-a82d-f4454df5a675 None admin] RBAC: Adding query filter params (interface=admin, service_id=5a9d453e0f0746fbb94458d940d04ceb, region_id=RegionOne) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:35:59.755299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c8d1a-3aff-4c78-a82d-f4454df5a675 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:59.756822 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c8d1a-3aff-4c78-a82d-f4454df5a675 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:59.762471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 496/988] 10.222.0.47 () {56 vars in 1249 bytes} [Thu Jul 14 15:35:59 2022] GET /identity/v3/endpoints?service_id=5a9d453e0f0746fbb94458d940d04ceb&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:35:59.767923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f5765-12ec-4a50-97a7-238a22bdd761 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:35:59.874550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f5765-12ec-4a50-97a7-238a22bdd761 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:35:59.908851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976f750-414f-4f99-84d0-93e6502c9c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:35:59.909517 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976f750-414f-4f99-84d0-93e6502c9c19 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:35:59.909804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976f750-414f-4f99-84d0-93e6502c9c19 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:35:59.910131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976f750-414f-4f99-84d0-93e6502c9c19 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:35:59.910817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976f750-414f-4f99-84d0-93e6502c9c19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:35:59.912191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976f750-414f-4f99-84d0-93e6502c9c19 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:35:59.918591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 493/989] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 14 15:35:59 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:00.190112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfc724-6461-4c7f-bbd9-bf1e201adcab None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:00.190557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfc724-6461-4c7f-bbd9-bf1e201adcab None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:00.190854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfc724-6461-4c7f-bbd9-bf1e201adcab None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:00.192369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 497/990] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:00.197304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332ddea-fe2d-405b-8d5f-12e7f2422837 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:00.197728 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332ddea-fe2d-405b-8d5f-12e7f2422837 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:00.197963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332ddea-fe2d-405b-8d5f-12e7f2422837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:00.215451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0332ddea-fe2d-405b-8d5f-12e7f2422837 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:00.216653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0332ddea-fe2d-405b-8d5f-12e7f2422837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:00.232185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:00.287808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 494/991] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:00.310523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfc724-6461-4c7f-bbd9-bf1e201adcab None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:00.347250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:00.414949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfc724-6461-4c7f-bbd9-bf1e201adcab None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:00.449713 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c14e5f-16ef-4b01-95af-e97b53d60a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:00.450452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c14e5f-16ef-4b01-95af-e97b53d60a8a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:00.450736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c14e5f-16ef-4b01-95af-e97b53d60a8a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:00.451011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c14e5f-16ef-4b01-95af-e97b53d60a8a None admin] PATH_INFO: `/v3/services/image` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:00.451663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c14e5f-16ef-4b01-95af-e97b53d60a8a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:00.453020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c14e5f-16ef-4b01-95af-e97b53d60a8a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-70c14e5f-16ef-4b01-95af-e97b53d60a8a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:00.464521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:00.465527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:00.472225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 498/992] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Jul 14 15:36:00 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:00.473133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332ddea-fe2d-405b-8d5f-12e7f2422837 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:00.571747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332ddea-fe2d-405b-8d5f-12e7f2422837 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:00.606758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b09ef0-62a7-4e0c-8b0b-10550e31b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:00.607432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b09ef0-62a7-4e0c-8b0b-10550e31b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:00.607711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b09ef0-62a7-4e0c-8b0b-10550e31b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:00.607984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b09ef0-62a7-4e0c-8b0b-10550e31b4d2 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:00.608701 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b09ef0-62a7-4e0c-8b0b-10550e31b4d2 None admin] RBAC: Adding query filter params (name=image) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:00.609009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b09ef0-62a7-4e0c-8b0b-10550e31b4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:00.610420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b09ef0-62a7-4e0c-8b0b-10550e31b4d2 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:00.620256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 495/993] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:00.621780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c14e5f-16ef-4b01-95af-e97b53d60a8a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:00.718557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:00.722919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c14e5f-16ef-4b01-95af-e97b53d60a8a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:00.751884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d008a3-1ac9-4ec7-9362-1fc6673d44d1 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:00.752178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d008a3-1ac9-4ec7-9362-1fc6673d44d1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:00.752437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d008a3-1ac9-4ec7-9362-1fc6673d44d1 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:00.753812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 496/994] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:36:00.757209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb1a7c-4aed-481a-9086-4edb47be6501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:00.758144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb1a7c-4aed-481a-9086-4edb47be6501 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:00.758655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb1a7c-4aed-481a-9086-4edb47be6501 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:00.758763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b965681-7e9a-41df-aa3f-3812ce6e6ee4 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:00.758989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b965681-7e9a-41df-aa3f-3812ce6e6ee4 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:00.759271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b965681-7e9a-41df-aa3f-3812ce6e6ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:00.759622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb1a7c-4aed-481a-9086-4edb47be6501 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:00.760532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb1a7c-4aed-481a-9086-4edb47be6501 None admin] RBAC: Adding query filter params (type=image) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:00.761203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb1a7c-4aed-481a-9086-4edb47be6501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:00.763284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb1a7c-4aed-481a-9086-4edb47be6501 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:00.769033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 499/995] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:00.782013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5b965681-7e9a-41df-aa3f-3812ce6e6ee4 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:00.783290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5b965681-7e9a-41df-aa3f-3812ce6e6ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:00.795224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:00.860168 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 497/996] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:36:00.883783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb1a7c-4aed-481a-9086-4edb47be6501 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:00.989087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb1a7c-4aed-481a-9086-4edb47be6501 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:01.024657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e96c18-087d-4f0e-b225-aad3c912d7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:01.025499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e96c18-087d-4f0e-b225-aad3c912d7aa None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:01.025784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e96c18-087d-4f0e-b225-aad3c912d7aa None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:01.026092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e96c18-087d-4f0e-b225-aad3c912d7aa None admin] PATH_INFO: `/v3/services/compute` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:01.026792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e96c18-087d-4f0e-b225-aad3c912d7aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:01.028727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e96c18-087d-4f0e-b225-aad3c912d7aa None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-84e96c18-087d-4f0e-b225-aad3c912d7aa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:01.041790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:01.043048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:01.048588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 500/997] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:36:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:01.050830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b965681-7e9a-41df-aa3f-3812ce6e6ee4 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:01.154097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b965681-7e9a-41df-aa3f-3812ce6e6ee4 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:01.188126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec3b49-e96d-4de8-a5d4-d0ee7f34e219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:01.188827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec3b49-e96d-4de8-a5d4-d0ee7f34e219 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:01.189117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec3b49-e96d-4de8-a5d4-d0ee7f34e219 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:01.189390 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec3b49-e96d-4de8-a5d4-d0ee7f34e219 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:01.190121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec3b49-e96d-4de8-a5d4-d0ee7f34e219 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:01.190705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec3b49-e96d-4de8-a5d4-d0ee7f34e219 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:01.192050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec3b49-e96d-4de8-a5d4-d0ee7f34e219 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:01.197074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 498/998] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:36:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:01.203134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e96c18-087d-4f0e-b225-aad3c912d7aa None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:01.231946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:01.305950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e96c18-087d-4f0e-b225-aad3c912d7aa None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:01.341107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779b94e-0ad1-429a-8ad5-a058676a5f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:01.341790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779b94e-0ad1-429a-8ad5-a058676a5f28 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:01.342106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779b94e-0ad1-429a-8ad5-a058676a5f28 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:01.342385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779b94e-0ad1-429a-8ad5-a058676a5f28 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:01.343096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779b94e-0ad1-429a-8ad5-a058676a5f28 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:01.343405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779b94e-0ad1-429a-8ad5-a058676a5f28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:01.344762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779b94e-0ad1-429a-8ad5-a058676a5f28 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:01.354584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 501/999] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:36:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:01.356174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec3b49-e96d-4de8-a5d4-d0ee7f34e219 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:01.367668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:01.461388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec3b49-e96d-4de8-a5d4-d0ee7f34e219 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:01.497080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9517eea-cd21-4a8f-9e04-9bd24a4bb4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:01.497744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9517eea-cd21-4a8f-9e04-9bd24a4bb4c5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:01.498057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9517eea-cd21-4a8f-9e04-9bd24a4bb4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:01.498335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9517eea-cd21-4a8f-9e04-9bd24a4bb4c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:01.499143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9517eea-cd21-4a8f-9e04-9bd24a4bb4c5 None admin] RBAC: Adding query filter params (interface=public, service_id=29595252ad2b45cbafa6216e59d9ef1c, region_id=RegionOne) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:01.499458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9517eea-cd21-4a8f-9e04-9bd24a4bb4c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:01.500826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9517eea-cd21-4a8f-9e04-9bd24a4bb4c5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:01.510667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 499/1000] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 14 15:36:01 2022] GET /identity/v3/endpoints?service_id=29595252ad2b45cbafa6216e59d9ef1c&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:01.511931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779b94e-0ad1-429a-8ad5-a058676a5f28 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:01.605796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:01.615398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779b94e-0ad1-429a-8ad5-a058676a5f28 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:01.649804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3711131-e13d-49f2-bd02-0d9c2f722aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:01.650557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3711131-e13d-49f2-bd02-0d9c2f722aa0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:01.650785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3711131-e13d-49f2-bd02-0d9c2f722aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:01.651064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3711131-e13d-49f2-bd02-0d9c2f722aa0 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:01.651748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3711131-e13d-49f2-bd02-0d9c2f722aa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:01.653625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3711131-e13d-49f2-bd02-0d9c2f722aa0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:01.661993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 502/1001] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 14 15:36:01 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:36:01.962620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52ea61-0c78-4431-8412-2e11cc9c9ed9 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:01.962952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52ea61-0c78-4431-8412-2e11cc9c9ed9 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:01.963196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52ea61-0c78-4431-8412-2e11cc9c9ed9 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:01.965252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 500/1002] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:01.970438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf9e3-4247-4b00-b04f-1ba85dce8446 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:01.970733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf9e3-4247-4b00-b04f-1ba85dce8446 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:01.971015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf9e3-4247-4b00-b04f-1ba85dce8446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:01.987648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-27dbf9e3-4247-4b00-b04f-1ba85dce8446 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:01.988853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-27dbf9e3-4247-4b00-b04f-1ba85dce8446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:02.064550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 503/1003] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:02.088404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52ea61-0c78-4431-8412-2e11cc9c9ed9 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:02.167622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:02.198075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52ea61-0c78-4431-8412-2e11cc9c9ed9 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:02.233745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69460206-12a3-49f0-a953-c27f3ee659bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:02.234503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69460206-12a3-49f0-a953-c27f3ee659bf None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:02.234788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69460206-12a3-49f0-a953-c27f3ee659bf None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:02.235067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69460206-12a3-49f0-a953-c27f3ee659bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:02.235726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69460206-12a3-49f0-a953-c27f3ee659bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:02.237105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69460206-12a3-49f0-a953-c27f3ee659bf None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-69460206-12a3-49f0-a953-c27f3ee659bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:02.249144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:02.250189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 15:36:02.251051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:02.252590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 501/1004] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 14 15:36:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:02.256360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf9e3-4247-4b00-b04f-1ba85dce8446 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:02.343890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:02.359990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf9e3-4247-4b00-b04f-1ba85dce8446 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:02.394997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1f481-a17c-4130-bdc6-718a5f09734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:02.395672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1f481-a17c-4130-bdc6-718a5f09734c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:02.395956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1f481-a17c-4130-bdc6-718a5f09734c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:02.396234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b1f481-a17c-4130-bdc6-718a5f09734c None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:02.396949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1f481-a17c-4130-bdc6-718a5f09734c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:02.397259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1f481-a17c-4130-bdc6-718a5f09734c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:02.398652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b1f481-a17c-4130-bdc6-718a5f09734c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:02.408696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 504/1005] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:36:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:02.410021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69460206-12a3-49f0-a953-c27f3ee659bf None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:02.512750 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69460206-12a3-49f0-a953-c27f3ee659bf None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:02.545539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acf495-825e-4149-bd3b-c6c77dcdc9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:02.546232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acf495-825e-4149-bd3b-c6c77dcdc9a8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:02.546521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acf495-825e-4149-bd3b-c6c77dcdc9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:02.546800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acf495-825e-4149-bd3b-c6c77dcdc9a8 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:02.547513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acf495-825e-4149-bd3b-c6c77dcdc9a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:02.547826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acf495-825e-4149-bd3b-c6c77dcdc9a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:02.549193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5acf495-825e-4149-bd3b-c6c77dcdc9a8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:02.551288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:02.555218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 502/1006] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 14 15:36:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:02.561355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1f481-a17c-4130-bdc6-718a5f09734c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:02.663645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b1f481-a17c-4130-bdc6-718a5f09734c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:02.698736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f10754-69b2-448b-ad1a-aa77b7b12958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:02.699431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f10754-69b2-448b-ad1a-aa77b7b12958 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:02.699718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f10754-69b2-448b-ad1a-aa77b7b12958 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:02.699997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f10754-69b2-448b-ad1a-aa77b7b12958 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:02.700752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f10754-69b2-448b-ad1a-aa77b7b12958 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:02.702144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f10754-69b2-448b-ad1a-aa77b7b12958 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:02.802153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca1ede-824c-4a00-9fdd-aed3d8f893d8 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:02.802469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca1ede-824c-4a00-9fdd-aed3d8f893d8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:02.802839 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca1ede-824c-4a00-9fdd-aed3d8f893d8 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:02.804392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 503/1007] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:36:02.809924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6a940-aa80-480f-be7e-2fd99ed7b575 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:02.810218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6a940-aa80-480f-be7e-2fd99ed7b575 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:02.810499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6a940-aa80-480f-be7e-2fd99ed7b575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:02.827459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a9c6a940-aa80-480f-be7e-2fd99ed7b575 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:02.828659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a9c6a940-aa80-480f-be7e-2fd99ed7b575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:02.852558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 505/1008] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 14 15:36:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:36:02.910763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 504/1009] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:36:02.934660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f10754-69b2-448b-ad1a-aa77b7b12958 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:02.946336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:03.047043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f10754-69b2-448b-ad1a-aa77b7b12958 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:03.083725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28bdb8-a207-49b7-a8df-7a0becedb90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:03.084324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28bdb8-a207-49b7-a8df-7a0becedb90b None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:03.084606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28bdb8-a207-49b7-a8df-7a0becedb90b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:03.084876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28bdb8-a207-49b7-a8df-7a0becedb90b None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:03.085605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28bdb8-a207-49b7-a8df-7a0becedb90b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:03.086996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28bdb8-a207-49b7-a8df-7a0becedb90b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:03.178378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 506/1010] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:36:02 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:03.713866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e3d7a-598a-4c8a-b7af-3709568ec757 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:03.714414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e3d7a-598a-4c8a-b7af-3709568ec757 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:03.714414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e3d7a-598a-4c8a-b7af-3709568ec757 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:03.715795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 505/1011] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:03.720847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004e0f4-b8c9-4da7-9c03-aeb94b528813 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:03.721243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004e0f4-b8c9-4da7-9c03-aeb94b528813 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:03.721411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004e0f4-b8c9-4da7-9c03-aeb94b528813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:03.738595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8004e0f4-b8c9-4da7-9c03-aeb94b528813 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:03.739781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8004e0f4-b8c9-4da7-9c03-aeb94b528813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:03.802000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:03.813988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 507/1012] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:03 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:03.838382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e3d7a-598a-4c8a-b7af-3709568ec757 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:03.943111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e3d7a-598a-4c8a-b7af-3709568ec757 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:03.969455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:03.978661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf663a-f630-41ad-8092-68d8060c32cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:03.979461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf663a-f630-41ad-8092-68d8060c32cb None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:03.979751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf663a-f630-41ad-8092-68d8060c32cb None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:03.980163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf663a-f630-41ad-8092-68d8060c32cb None admin] PATH_INFO: `/v3/services/compute` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:03.981458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf663a-f630-41ad-8092-68d8060c32cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:03.982883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf663a-f630-41ad-8092-68d8060c32cb None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-f4cf663a-f630-41ad-8092-68d8060c32cb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:03.995853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:03.996939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 15:36:03.998251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:04.002170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 506/1013] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:36:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:04.003496 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004e0f4-b8c9-4da7-9c03-aeb94b528813 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:04.113242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004e0f4-b8c9-4da7-9c03-aeb94b528813 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:04.146542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a5d2c-b6bd-4e19-9c07-42db5c7ed949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:04.147222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a5d2c-b6bd-4e19-9c07-42db5c7ed949 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:04.147507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a5d2c-b6bd-4e19-9c07-42db5c7ed949 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:04.147785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a5d2c-b6bd-4e19-9c07-42db5c7ed949 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:04.148495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a5d2c-b6bd-4e19-9c07-42db5c7ed949 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:04.148803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a5d2c-b6bd-4e19-9c07-42db5c7ed949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:04.150187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a5d2c-b6bd-4e19-9c07-42db5c7ed949 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:04.160198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 508/1014] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:36:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:04.161364 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf663a-f630-41ad-8092-68d8060c32cb None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:04.262676 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf663a-f630-41ad-8092-68d8060c32cb None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:04.297256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72017716-4e78-4598-a706-476c0fbc0557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:04.297981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72017716-4e78-4598-a706-476c0fbc0557 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:04.298271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72017716-4e78-4598-a706-476c0fbc0557 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:04.298548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72017716-4e78-4598-a706-476c0fbc0557 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:04.299283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72017716-4e78-4598-a706-476c0fbc0557 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:04.299603 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72017716-4e78-4598-a706-476c0fbc0557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:04.301017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72017716-4e78-4598-a706-476c0fbc0557 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:04.311385 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 507/1015] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 14 15:36:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:04.312410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a5d2c-b6bd-4e19-9c07-42db5c7ed949 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:04.373355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:04.416069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a5d2c-b6bd-4e19-9c07-42db5c7ed949 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:04.450762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955305a-04c7-4665-997f-81196d6fb8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:04.451431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955305a-04c7-4665-997f-81196d6fb8ff None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:04.451718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955305a-04c7-4665-997f-81196d6fb8ff None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:04.451997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955305a-04c7-4665-997f-81196d6fb8ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:04.452741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955305a-04c7-4665-997f-81196d6fb8ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:04.454133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955305a-04c7-4665-997f-81196d6fb8ff None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:04.529944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 509/1016] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 14 15:36:04 2022] POST /identity/v3/endpoints => generated 334 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:36:04.932857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f632c77-f5b8-4f8c-9884-c82c2cac81f1 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:04.933863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f632c77-f5b8-4f8c-9884-c82c2cac81f1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:04.933863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f632c77-f5b8-4f8c-9884-c82c2cac81f1 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:04.935380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 508/1017] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:04.940712 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882a16a-3f65-4ec3-b4ba-ced94339193e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:04.941100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882a16a-3f65-4ec3-b4ba-ced94339193e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:04.941264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882a16a-3f65-4ec3-b4ba-ced94339193e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:04.958085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b882a16a-3f65-4ec3-b4ba-ced94339193e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:04.959263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b882a16a-3f65-4ec3-b4ba-ced94339193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:05.030954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 510/1018] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:05.055957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f632c77-f5b8-4f8c-9884-c82c2cac81f1 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:05.116139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:05.163451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f632c77-f5b8-4f8c-9884-c82c2cac81f1 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:05.198550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfe6eb-90b0-45a9-a9f6-5a7205f845c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:05.199318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfe6eb-90b0-45a9-a9f6-5a7205f845c2 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:05.199603 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfe6eb-90b0-45a9-a9f6-5a7205f845c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:05.199880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfe6eb-90b0-45a9-a9f6-5a7205f845c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:05.201033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfe6eb-90b0-45a9-a9f6-5a7205f845c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:05.202503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfe6eb-90b0-45a9-a9f6-5a7205f845c2 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:05.204307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 509/1019] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:05.210015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882a16a-3f65-4ec3-b4ba-ced94339193e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:05.231726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0b3ed-e319-4184-864a-12bad2e958df None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:05.232020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0b3ed-e319-4184-864a-12bad2e958df None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:05.232301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0b3ed-e319-4184-864a-12bad2e958df None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:05.233857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 510/1020] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:36:05.238666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a5ae6-3a95-4c74-898d-73f3be98a396 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:05.238956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a5ae6-3a95-4c74-898d-73f3be98a396 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:05.239591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a5ae6-3a95-4c74-898d-73f3be98a396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:05.256154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-750a5ae6-3a95-4c74-898d-73f3be98a396 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:05.257013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:05.257523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-750a5ae6-3a95-4c74-898d-73f3be98a396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:05.318378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882a16a-3f65-4ec3-b4ba-ced94339193e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:05.330741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 511/1021] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:36:05.354027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a5ae6-3a95-4c74-898d-73f3be98a396 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:05.356806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112aa87-f12f-4268-823e-2ea302404ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:05.357767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112aa87-f12f-4268-823e-2ea302404ae5 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:05.358088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112aa87-f12f-4268-823e-2ea302404ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:05.358370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112aa87-f12f-4268-823e-2ea302404ae5 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:05.359103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112aa87-f12f-4268-823e-2ea302404ae5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:05.360542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112aa87-f12f-4268-823e-2ea302404ae5 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:05.408594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:05.440890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 511/1022] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:36:05 2022] POST /identity/v3/projects => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:05.459678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a5ae6-3a95-4c74-898d-73f3be98a396 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:05.501091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40394fc-aefd-45e9-91da-fb0036cf1ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:05.501944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40394fc-aefd-45e9-91da-fb0036cf1ed2 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:05.502227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40394fc-aefd-45e9-91da-fb0036cf1ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:05.502507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40394fc-aefd-45e9-91da-fb0036cf1ed2 None admin] PATH_INFO: `/v3/services/image` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:05.503185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40394fc-aefd-45e9-91da-fb0036cf1ed2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:05.504601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40394fc-aefd-45e9-91da-fb0036cf1ed2 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-b40394fc-aefd-45e9-91da-fb0036cf1ed2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:05.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:05.518718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:05.519578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 512/1023] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Jul 14 15:36:05 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:05.525881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112aa87-f12f-4268-823e-2ea302404ae5 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:05.624871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:05.633935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112aa87-f12f-4268-823e-2ea302404ae5 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:05.667984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb068c-485b-48a8-abec-193c0069a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:05.668662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb068c-485b-48a8-abec-193c0069a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:05.668948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb068c-485b-48a8-abec-193c0069a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:05.669225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb068c-485b-48a8-abec-193c0069a3e2 None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:05.669965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb068c-485b-48a8-abec-193c0069a3e2 None admin] RBAC: Adding query filter params (name=image) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:05.670276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb068c-485b-48a8-abec-193c0069a3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:05.671688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb068c-485b-48a8-abec-193c0069a3e2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:05.676681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 512/1024] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:05.682854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40394fc-aefd-45e9-91da-fb0036cf1ed2 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:05.791475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40394fc-aefd-45e9-91da-fb0036cf1ed2 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:05.807710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:05.827046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6025382-da5c-46a3-880e-b2f3d403ed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:05.827724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6025382-da5c-46a3-880e-b2f3d403ed9d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:05.828011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6025382-da5c-46a3-880e-b2f3d403ed9d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:05.828286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6025382-da5c-46a3-880e-b2f3d403ed9d None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:05.828994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6025382-da5c-46a3-880e-b2f3d403ed9d None admin] RBAC: Adding query filter params (type=image) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:05.829302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6025382-da5c-46a3-880e-b2f3d403ed9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:05.830752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6025382-da5c-46a3-880e-b2f3d403ed9d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:05.835944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 513/1025] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:05.843112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb068c-485b-48a8-abec-193c0069a3e2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:05.942878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb068c-485b-48a8-abec-193c0069a3e2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:05.977439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe585cb-9cad-46d2-b9e3-8af2b47e5c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:05.978147 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe585cb-9cad-46d2-b9e3-8af2b47e5c50 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:05.978438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe585cb-9cad-46d2-b9e3-8af2b47e5c50 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:05.978720 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe585cb-9cad-46d2-b9e3-8af2b47e5c50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:05.979510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe585cb-9cad-46d2-b9e3-8af2b47e5c50 None admin] RBAC: Adding query filter params (interface=public, service_id=88279978439d4b2ca49fbfd0cf4b9cc6, region_id=RegionOne) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:05.979826 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe585cb-9cad-46d2-b9e3-8af2b47e5c50 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:05.981244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe585cb-9cad-46d2-b9e3-8af2b47e5c50 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:05.991103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 513/1026] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 14 15:36:05 2022] GET /identity/v3/endpoints?service_id=88279978439d4b2ca49fbfd0cf4b9cc6&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:05.992070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6025382-da5c-46a3-880e-b2f3d403ed9d None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:06.103694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6025382-da5c-46a3-880e-b2f3d403ed9d None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:06.137791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b819cf7-a9d2-45a9-bdf2-f090251836c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:06.138492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b819cf7-a9d2-45a9-bdf2-f090251836c5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:06.138779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b819cf7-a9d2-45a9-bdf2-f090251836c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:06.139056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b819cf7-a9d2-45a9-bdf2-f090251836c5 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:06.139740 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b819cf7-a9d2-45a9-bdf2-f090251836c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:06.141117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b819cf7-a9d2-45a9-bdf2-f090251836c5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:06.153907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 514/1027] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 14 15:36:05 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:36:06.678238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc4e2e-2c21-4af3-a135-fb16356c790b None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:06.678591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc4e2e-2c21-4af3-a135-fb16356c790b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:06.678809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc4e2e-2c21-4af3-a135-fb16356c790b None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:06.680247 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 514/1028] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:06.685461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5babec-cc15-4c61-8fc0-9870ae40a14e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:06.685761 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5babec-cc15-4c61-8fc0-9870ae40a14e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:06.686072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5babec-cc15-4c61-8fc0-9870ae40a14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:06.703656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ad5babec-cc15-4c61-8fc0-9870ae40a14e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:06.704850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ad5babec-cc15-4c61-8fc0-9870ae40a14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:06.745406 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:06.774056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 515/1029] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:06.796459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc4e2e-2c21-4af3-a135-fb16356c790b None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:06.818974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:06.901063 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc4e2e-2c21-4af3-a135-fb16356c790b None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:06.935129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c99ed-f7ba-4733-b043-4bb2e30868ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:06.935875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c99ed-f7ba-4733-b043-4bb2e30868ce None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:06.936161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c99ed-f7ba-4733-b043-4bb2e30868ce None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:06.936439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c99ed-f7ba-4733-b043-4bb2e30868ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:06.947358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c99ed-f7ba-4733-b043-4bb2e30868ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-331c99ed-f7ba-4733-b043-4bb2e30868ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:06.955442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:06.956482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:36:06.957330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:06.961767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 515/1030] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 14 15:36:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:06.962797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5babec-cc15-4c61-8fc0-9870ae40a14e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:07.069181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5babec-cc15-4c61-8fc0-9870ae40a14e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:07.104464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7903e26-fd49-4336-969d-8082735793e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:07.105200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7903e26-fd49-4336-969d-8082735793e1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.105484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7903e26-fd49-4336-969d-8082735793e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.105763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7903e26-fd49-4336-969d-8082735793e1 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.106538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7903e26-fd49-4336-969d-8082735793e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:07.106850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7903e26-fd49-4336-969d-8082735793e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:07.108222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7903e26-fd49-4336-969d-8082735793e1 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:07.116781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 516/1031] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 14 15:36:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:07.122945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c99ed-f7ba-4733-b043-4bb2e30868ce None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:07.205113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:07.224678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c99ed-f7ba-4733-b043-4bb2e30868ce None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:07.259942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e5f3b-7c72-43a9-b9d5-ec9456f71bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:07.260743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e5f3b-7c72-43a9-b9d5-ec9456f71bfc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.261030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e5f3b-7c72-43a9-b9d5-ec9456f71bfc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.261880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e5f3b-7c72-43a9-b9d5-ec9456f71bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.273605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e5f3b-7c72-43a9-b9d5-ec9456f71bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:07.275143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e5f3b-7c72-43a9-b9d5-ec9456f71bfc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-a82e5f3b-7c72-43a9-b9d5-ec9456f71bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:36:07.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:07.295607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:07.296528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:36:07.296528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:36:07.296528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:07.296528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:07.300634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 516/1032] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:07.302032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7903e26-fd49-4336-969d-8082735793e1 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:07.338616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:07.405647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7903e26-fd49-4336-969d-8082735793e1 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:07.439756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413353cf-0333-49e3-9db4-abff8ada5b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:07.440681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413353cf-0333-49e3-9db4-abff8ada5b19 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.440964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413353cf-0333-49e3-9db4-abff8ada5b19 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.441240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413353cf-0333-49e3-9db4-abff8ada5b19 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.441995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413353cf-0333-49e3-9db4-abff8ada5b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:07.442308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413353cf-0333-49e3-9db4-abff8ada5b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:07.443674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413353cf-0333-49e3-9db4-abff8ada5b19 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:07.456391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 517/1033] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:07.461857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e5f3b-7c72-43a9-b9d5-ec9456f71bfc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:07.533660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddedeed-e42c-4371-ac8b-c5699e507dc8 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.533988 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddedeed-e42c-4371-ac8b-c5699e507dc8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.534270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddedeed-e42c-4371-ac8b-c5699e507dc8 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.535643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 518/1034] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:36:07.540577 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584032ff-0033-47f7-aefd-b598cdb379db None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.540863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584032ff-0033-47f7-aefd-b598cdb379db None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.541145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584032ff-0033-47f7-aefd-b598cdb379db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.558553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-584032ff-0033-47f7-aefd-b598cdb379db None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:07.559832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-584032ff-0033-47f7-aefd-b598cdb379db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:07.567965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e5f3b-7c72-43a9-b9d5-ec9456f71bfc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:07.597967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:07.602917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd3872-8997-406c-9627-a46616d694e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:07.603824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd3872-8997-406c-9627-a46616d694e3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.604113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd3872-8997-406c-9627-a46616d694e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.604388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd3872-8997-406c-9627-a46616d694e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.612599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd3872-8997-406c-9627-a46616d694e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:07.614063 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd3872-8997-406c-9627-a46616d694e3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-24bd3872-8997-406c-9627-a46616d694e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:36:07.628525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:07.629698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:07.630744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 517/1035] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 14 15:36:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:07.635959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd3872-8997-406c-9627-a46616d694e3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:07.637174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 519/1036] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:36:07.661919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584032ff-0033-47f7-aefd-b598cdb379db None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:07.689342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:07.744561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd3872-8997-406c-9627-a46616d694e3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:07.768105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584032ff-0033-47f7-aefd-b598cdb379db None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:07.777835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99aed-61de-41da-a088-dfa63ae686b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:07.778522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99aed-61de-41da-a088-dfa63ae686b2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.778806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99aed-61de-41da-a088-dfa63ae686b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.779084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99aed-61de-41da-a088-dfa63ae686b2 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.779919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a99aed-61de-41da-a088-dfa63ae686b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:07.780232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a99aed-61de-41da-a088-dfa63ae686b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:07.781659 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a99aed-61de-41da-a088-dfa63ae686b2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:07.799549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 518/1037] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 14 15:36:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:07.805093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99aed-61de-41da-a088-dfa63ae686b2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:07.812012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa504037-aa0b-48a4-a9ba-5fe33bcd6464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:07.813033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa504037-aa0b-48a4-a9ba-5fe33bcd6464 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.813486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa504037-aa0b-48a4-a9ba-5fe33bcd6464 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.813761 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa504037-aa0b-48a4-a9ba-5fe33bcd6464 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.815008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa504037-aa0b-48a4-a9ba-5fe33bcd6464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:07.816697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa504037-aa0b-48a4-a9ba-5fe33bcd6464 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:07.818369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 520/1038] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:07.823857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa504037-aa0b-48a4-a9ba-5fe33bcd6464 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:07.860791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:07.911124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99aed-61de-41da-a088-dfa63ae686b2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:07.924230 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:07.932021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa504037-aa0b-48a4-a9ba-5fe33bcd6464 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:07.947028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58395f0e-5c4f-4339-b29b-ca88f4f8e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:07.947978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58395f0e-5c4f-4339-b29b-ca88f4f8e157 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.948884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58395f0e-5c4f-4339-b29b-ca88f4f8e157 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.949174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58395f0e-5c4f-4339-b29b-ca88f4f8e157 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.969791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3b9ed-9d93-432e-8c2a-b5d6c68d4d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:07.970486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3b9ed-9d93-432e-8c2a-b5d6c68d4d22 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:07.970768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3b9ed-9d93-432e-8c2a-b5d6c68d4d22 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:07.971040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3b9ed-9d93-432e-8c2a-b5d6c68d4d22 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:07.971848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3b9ed-9d93-432e-8c2a-b5d6c68d4d22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:07.973278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3b9ed-9d93-432e-8c2a-b5d6c68d4d22 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.018412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58395f0e-5c4f-4339-b29b-ca88f4f8e157 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.019952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58395f0e-5c4f-4339-b29b-ca88f4f8e157 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.068968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 521/1039] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:36:07 2022] POST /identity/v3/users => generated 285 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:08.076481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:08.138629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 519/1040] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 15:36:08.140316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3b9ed-9d93-432e-8c2a-b5d6c68d4d22 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:08.217578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae8868-9340-43d2-af5f-1f2b98a1d199 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.217910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae8868-9340-43d2-af5f-1f2b98a1d199 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.218204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae8868-9340-43d2-af5f-1f2b98a1d199 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.219598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 520/1041] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:08.224841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b27c88-7e86-48ce-9e06-770d40eb2c7e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.225130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b27c88-7e86-48ce-9e06-770d40eb2c7e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.225408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b27c88-7e86-48ce-9e06-770d40eb2c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.243282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a0b27c88-7e86-48ce-9e06-770d40eb2c7e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:08.244504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a0b27c88-7e86-48ce-9e06-770d40eb2c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:08.253704 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3b9ed-9d93-432e-8c2a-b5d6c68d4d22 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:08.297595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2b3a5-e55e-4d94-9789-232eb1c28f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:08.298365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2b3a5-e55e-4d94-9789-232eb1c28f62 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.298651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2b3a5-e55e-4d94-9789-232eb1c28f62 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.298928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2b3a5-e55e-4d94-9789-232eb1c28f62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.311308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2b3a5-e55e-4d94-9789-232eb1c28f62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.312779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2b3a5-e55e-4d94-9789-232eb1c28f62 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.326699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 521/1042] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-93f2b3a5-e55e-4d94-9789-232eb1c28f62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:36:08.331195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:08.332156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:08.333106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:36:08.333106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:36:08.333106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:08.333106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:08.333890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 522/1043] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:08.338284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b27c88-7e86-48ce-9e06-770d40eb2c7e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:08.350004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2b3a5-e55e-4d94-9789-232eb1c28f62 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:08.353405 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:08.425580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:08.445795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b27c88-7e86-48ce-9e06-770d40eb2c7e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:08.454648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2b3a5-e55e-4d94-9789-232eb1c28f62 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:08.485948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f01b9-9337-41fa-aaa8-e93f02203284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:08.486634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f01b9-9337-41fa-aaa8-e93f02203284 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.487153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f01b9-9337-41fa-aaa8-e93f02203284 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.487423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f01b9-9337-41fa-aaa8-e93f02203284 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.488271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f01b9-9337-41fa-aaa8-e93f02203284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:08.488536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f01b9-9337-41fa-aaa8-e93f02203284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.490033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1f01b9-9337-41fa-aaa8-e93f02203284 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.491341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da391035-b06e-4319-bb9c-7c7cb3b45891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:08.492143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da391035-b06e-4319-bb9c-7c7cb3b45891 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.492427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da391035-b06e-4319-bb9c-7c7cb3b45891 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.492711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da391035-b06e-4319-bb9c-7c7cb3b45891 None admin] PATH_INFO: `/v3/services/image` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.493370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da391035-b06e-4319-bb9c-7c7cb3b45891 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.494796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da391035-b06e-4319-bb9c-7c7cb3b45891 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.502986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 522/1044] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-da391035-b06e-4319-bb9c-7c7cb3b45891 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:08.507450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:08.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 15:36:08.509319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:08.510336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f01b9-9337-41fa-aaa8-e93f02203284 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:08.513758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 523/1045] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Jul 14 15:36:08 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:08.514816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da391035-b06e-4319-bb9c-7c7cb3b45891 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:08.583507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:08.616371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1f01b9-9337-41fa-aaa8-e93f02203284 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:08.621465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da391035-b06e-4319-bb9c-7c7cb3b45891 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:08.650729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383658e-ccef-4081-83fd-df4d9f0345d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:08.651541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383658e-ccef-4081-83fd-df4d9f0345d3 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.652360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383658e-ccef-4081-83fd-df4d9f0345d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.652649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383658e-ccef-4081-83fd-df4d9f0345d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.657003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52631fc3-266b-46c0-89d6-969aef978e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:08.657883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52631fc3-266b-46c0-89d6-969aef978e8c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.658315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52631fc3-266b-46c0-89d6-969aef978e8c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.658749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52631fc3-266b-46c0-89d6-969aef978e8c None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.659464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52631fc3-266b-46c0-89d6-969aef978e8c None admin] RBAC: Adding query filter params (name=image) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:08.659772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52631fc3-266b-46c0-89d6-969aef978e8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.659876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383658e-ccef-4081-83fd-df4d9f0345d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.661144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52631fc3-266b-46c0-89d6-969aef978e8c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.661245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383658e-ccef-4081-83fd-df4d9f0345d3 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.664297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:08.671341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 524/1046] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:08.677065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52631fc3-266b-46c0-89d6-969aef978e8c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-7383658e-ccef-4081-83fd-df4d9f0345d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:36:08.680136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:08.681008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:08.682088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:36:08.682088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:36:08.682088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:36:08.682088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:36:08.682088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:36:08.682088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:36:08.682088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:36:08.682088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:08.682088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 523/1047] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:08.687275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383658e-ccef-4081-83fd-df4d9f0345d3 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:08.785153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52631fc3-266b-46c0-89d6-969aef978e8c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:08.791648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383658e-ccef-4081-83fd-df4d9f0345d3 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:08.814924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:08.822345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e55ba0-befd-49f5-9f21-d48befbf4708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:08.823013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e55ba0-befd-49f5-9f21-d48befbf4708 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.823296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e55ba0-befd-49f5-9f21-d48befbf4708 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.831399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e55ba0-befd-49f5-9f21-d48befbf4708 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.831399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e55ba0-befd-49f5-9f21-d48befbf4708 None admin] RBAC: Adding query filter params (type=image) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:08.831399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e55ba0-befd-49f5-9f21-d48befbf4708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.831399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e55ba0-befd-49f5-9f21-d48befbf4708 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.831704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcad0f8-69e2-4690-958c-403ac7f4b124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:08.831704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcad0f8-69e2-4690-958c-403ac7f4b124 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.831704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcad0f8-69e2-4690-958c-403ac7f4b124 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.831704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcad0f8-69e2-4690-958c-403ac7f4b124 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.831978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcad0f8-69e2-4690-958c-403ac7f4b124 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:08.831978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcad0f8-69e2-4690-958c-403ac7f4b124 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.832113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 525/1048] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:08.834313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcad0f8-69e2-4690-958c-403ac7f4b124 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.837043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e55ba0-befd-49f5-9f21-d48befbf4708 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:08.852037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 524/1049] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:36:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:08.853434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcad0f8-69e2-4690-958c-403ac7f4b124 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:08.915395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:08.952044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e55ba0-befd-49f5-9f21-d48befbf4708 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:08.958554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcad0f8-69e2-4690-958c-403ac7f4b124 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:08.988015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d540a3-9999-4ca7-a1b1-429ac128b5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:08.988674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d540a3-9999-4ca7-a1b1-429ac128b5ef None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.988955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d540a3-9999-4ca7-a1b1-429ac128b5ef None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.989221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d540a3-9999-4ca7-a1b1-429ac128b5ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.989999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d540a3-9999-4ca7-a1b1-429ac128b5ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.991403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d540a3-9999-4ca7-a1b1-429ac128b5ef None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:08.994266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741bb46-b373-438d-9f8e-3aceffc545a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:08.995082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741bb46-b373-438d-9f8e-3aceffc545a5 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:08.995697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741bb46-b373-438d-9f8e-3aceffc545a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:08.995981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741bb46-b373-438d-9f8e-3aceffc545a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:08.996813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741bb46-b373-438d-9f8e-3aceffc545a5 None admin] RBAC: Adding query filter params (role.id=5ccecebd4103442484b3dadfbfbf47c4, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=ce88c973feef4627a1b871ee9cb26511) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:08.997127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741bb46-b373-438d-9f8e-3aceffc545a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:08.998560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741bb46-b373-438d-9f8e-3aceffc545a5 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:09.004511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 525/1050] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:36:08 2022] GET /identity/v3/role_assignments?user.id=ce88c973feef4627a1b871ee9cb26511&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=5ccecebd4103442484b3dadfbfbf47c4 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:09.113949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 526/1051] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 14 15:36:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:36:10.169616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e84a9-6c62-4b18-877d-e4886b00c65c None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:10.170170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e84a9-6c62-4b18-877d-e4886b00c65c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:10.170283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e84a9-6c62-4b18-877d-e4886b00c65c None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:10.171631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 526/1052] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:10.176813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fb218-8ee5-4c88-894d-9c7b97afc16a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:10.177210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fb218-8ee5-4c88-894d-9c7b97afc16a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:10.177371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fb218-8ee5-4c88-894d-9c7b97afc16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:10.195012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ff6fb218-8ee5-4c88-894d-9c7b97afc16a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:10.196199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ff6fb218-8ee5-4c88-894d-9c7b97afc16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:10.246417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:10.273310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 527/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:10.297306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e84a9-6c62-4b18-877d-e4886b00c65c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:10.398879 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e84a9-6c62-4b18-877d-e4886b00c65c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:10.421902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:10.432783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93784ac9-6471-43d7-a4e6-4500260b94dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:10.433628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93784ac9-6471-43d7-a4e6-4500260b94dd None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:10.433940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93784ac9-6471-43d7-a4e6-4500260b94dd None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:10.434217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93784ac9-6471-43d7-a4e6-4500260b94dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:10.444855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93784ac9-6471-43d7-a4e6-4500260b94dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-93784ac9-6471-43d7-a4e6-4500260b94dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:10.453127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:10.454088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:10.454978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:10.457964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 527/1054] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:36:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:10.460612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fb218-8ee5-4c88-894d-9c7b97afc16a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:10.560929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fb218-8ee5-4c88-894d-9c7b97afc16a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:10.595917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb59f4-e67a-48f1-8624-88d747e9fc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:10.596602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb59f4-e67a-48f1-8624-88d747e9fc28 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:10.596888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb59f4-e67a-48f1-8624-88d747e9fc28 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:10.597174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb59f4-e67a-48f1-8624-88d747e9fc28 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:10.597958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb59f4-e67a-48f1-8624-88d747e9fc28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:10.598268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb59f4-e67a-48f1-8624-88d747e9fc28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:10.599637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb59f4-e67a-48f1-8624-88d747e9fc28 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:10.613506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 528/1055] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:36:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:10.614799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93784ac9-6471-43d7-a4e6-4500260b94dd None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:10.718844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93784ac9-6471-43d7-a4e6-4500260b94dd None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:10.754114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939da89-223d-493b-83c2-2f62c0bfa57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:10.754989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939da89-223d-493b-83c2-2f62c0bfa57d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:10.755269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939da89-223d-493b-83c2-2f62c0bfa57d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:10.755540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939da89-223d-493b-83c2-2f62c0bfa57d None admin] PATH_INFO: `/v3/users/81717922951641bda7ba774999b98f50` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:10.769080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939da89-223d-493b-83c2-2f62c0bfa57d None admin] RBAC: Authorizing `identity:get_user(user_id=81717922951641bda7ba774999b98f50)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:10.770545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939da89-223d-493b-83c2-2f62c0bfa57d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:10.777274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 528/1056] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:10 2022] GET /identity/v3/users/81717922951641bda7ba774999b98f50 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:10.778551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb59f4-e67a-48f1-8624-88d747e9fc28 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:10.821288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:10.882490 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb59f4-e67a-48f1-8624-88d747e9fc28 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:10.916879 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca6c3b-c8ac-4c4f-bf33-fe896bac7ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:10.917594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca6c3b-c8ac-4c4f-bf33-fe896bac7ef0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:10.917905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca6c3b-c8ac-4c4f-bf33-fe896bac7ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:10.918184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca6c3b-c8ac-4c4f-bf33-fe896bac7ef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:10.986252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca6c3b-c8ac-4c4f-bf33-fe896bac7ef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:10.987720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca6c3b-c8ac-4c4f-bf33-fe896bac7ef0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:11.026207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:11.027285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77175f-de7b-4497-8a65-3948169e549b None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.027583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77175f-de7b-4497-8a65-3948169e549b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.027865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77175f-de7b-4497-8a65-3948169e549b None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.031690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 529/1057] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:11.035791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7c0c-6c39-4a79-9c12-8bb5d1f1c203 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.035791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7c0c-6c39-4a79-9c12-8bb5d1f1c203 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.035791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7c0c-6c39-4a79-9c12-8bb5d1f1c203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.053186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-6bdf7c0c-6c39-4a79-9c12-8bb5d1f1c203 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:11.054422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6bdf7c0c-6c39-4a79-9c12-8bb5d1f1c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:11.093312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:11.102163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 529/1058] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:11.108475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca6c3b-c8ac-4c4f-bf33-fe896bac7ef0 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.134567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 530/1059] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:36:11.157250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf7c0c-6c39-4a79-9c12-8bb5d1f1c203 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.211403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca6c3b-c8ac-4c4f-bf33-fe896bac7ef0 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:11.245298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c0b03-d829-4ae8-b0f8-a52d5a7f1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:11.246735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c0b03-d829-4ae8-b0f8-a52d5a7f1564 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.247014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c0b03-d829-4ae8-b0f8-a52d5a7f1564 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.247283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c0b03-d829-4ae8-b0f8-a52d5a7f1564 None admin] PATH_INFO: `/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.248465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c0b03-d829-4ae8-b0f8-a52d5a7f1564 None admin] RBAC: Authorizing `identity:get_project(project_id=51cc6bd9b1d34d0d92a6589f9e7be953)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:11.249929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c0b03-d829-4ae8-b0f8-a52d5a7f1564 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:11.251618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 530/1060] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:11.252789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cf8e3-678a-4a61-acc3-eb6daace24c4 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.253071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cf8e3-678a-4a61-acc3-eb6daace24c4 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.253344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cf8e3-678a-4a61-acc3-eb6daace24c4 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.254583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 531/1061] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:11.258928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cf8e3-678a-4a61-acc3-eb6daace24c4 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.262347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf7c0c-6c39-4a79-9c12-8bb5d1f1c203 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:11.268094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:11.296463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba1914-ac38-4fdb-beda-9b76acf20dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:11.297274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba1914-ac38-4fdb-beda-9b76acf20dff None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.297561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba1914-ac38-4fdb-beda-9b76acf20dff None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.297864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba1914-ac38-4fdb-beda-9b76acf20dff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.309379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba1914-ac38-4fdb-beda-9b76acf20dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-d6ba1914-ac38-4fdb-beda-9b76acf20dff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:11.317773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:11.318751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:11.319610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 531/1062] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:11.320348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f81a69-cf76-462b-a8bf-6866e1b48795 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.320518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f81a69-cf76-462b-a8bf-6866e1b48795 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.320855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f81a69-cf76-462b-a8bf-6866e1b48795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.337196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-84f81a69-cf76-462b-a8bf-6866e1b48795 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:11.338436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-84f81a69-cf76-462b-a8bf-6866e1b48795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:11.363881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:11.364019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cf8e3-678a-4a61-acc3-eb6daace24c4 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:11.398417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880f54c-c71c-44b5-a00e-71325ee0221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:11.399072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880f54c-c71c-44b5-a00e-71325ee0221d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.399359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880f54c-c71c-44b5-a00e-71325ee0221d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.399641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880f54c-c71c-44b5-a00e-71325ee0221d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.400465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880f54c-c71c-44b5-a00e-71325ee0221d None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=51cc6bd9b1d34d0d92a6589f9e7be953, user.id=81717922951641bda7ba774999b98f50) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:11.400782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880f54c-c71c-44b5-a00e-71325ee0221d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:11.402754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880f54c-c71c-44b5-a00e-71325ee0221d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:11.404744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 532/1063] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:11.405743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f81a69-cf76-462b-a8bf-6866e1b48795 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.409181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 532/1064] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/role_assignments?user.id=81717922951641bda7ba774999b98f50&scope.project.id=51cc6bd9b1d34d0d92a6589f9e7be953&role.id=abfb60dccca7470295258595a66a5d17 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:11.427401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880f54c-c71c-44b5-a00e-71325ee0221d None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.507296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:11.524219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f81a69-cf76-462b-a8bf-6866e1b48795 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:11.542871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880f54c-c71c-44b5-a00e-71325ee0221d None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:11.564227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2c31f-c724-42e4-b8bc-38f66fa76db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:11.564918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2c31f-c724-42e4-b8bc-38f66fa76db4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.565743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2c31f-c724-42e4-b8bc-38f66fa76db4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.565743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2c31f-c724-42e4-b8bc-38f66fa76db4 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.567115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2c31f-c724-42e4-b8bc-38f66fa76db4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:11.567115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2c31f-c724-42e4-b8bc-38f66fa76db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:11.568181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2c31f-c724-42e4-b8bc-38f66fa76db4 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:11.578187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 533/1065] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:11.583367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25d538-b8fd-4146-85d7-d48ee2b51791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:11.583798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2c31f-c724-42e4-b8bc-38f66fa76db4 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.584093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25d538-b8fd-4146-85d7-d48ee2b51791 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.584374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25d538-b8fd-4146-85d7-d48ee2b51791 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.584655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25d538-b8fd-4146-85d7-d48ee2b51791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.615218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:11.658219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25d538-b8fd-4146-85d7-d48ee2b51791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:11.659687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe25d538-b8fd-4146-85d7-d48ee2b51791 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:11.698768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2c31f-c724-42e4-b8bc-38f66fa76db4 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:11.734127 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:11.735907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f9097-7bd6-4b61-b8ba-820d7dd291ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:11.736720 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f9097-7bd6-4b61-b8ba-820d7dd291ee None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.737009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f9097-7bd6-4b61-b8ba-820d7dd291ee None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.737283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f9097-7bd6-4b61-b8ba-820d7dd291ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.748705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f9097-7bd6-4b61-b8ba-820d7dd291ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:11.750191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f9097-7bd6-4b61-b8ba-820d7dd291ee None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:11.769293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 533/1066] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-253f9097-7bd6-4b61-b8ba-820d7dd291ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:36:11.769456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:11.770594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:11.771506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:36:11.771506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:36:11.771506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:11.771506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:11.775388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25d538-b8fd-4146-85d7-d48ee2b51791 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.775513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 534/1067] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:11.776106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f9097-7bd6-4b61-b8ba-820d7dd291ee None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.862850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:11.881538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe25d538-b8fd-4146-85d7-d48ee2b51791 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:11.885311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f9097-7bd6-4b61-b8ba-820d7dd291ee None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:11.918479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d014-c968-4f5b-a175-3b0a7f9d8459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:11.919414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d014-c968-4f5b-a175-3b0a7f9d8459 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.919706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d014-c968-4f5b-a175-3b0a7f9d8459 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.919987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d014-c968-4f5b-a175-3b0a7f9d8459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.922915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a129d-3838-4b6c-ba3c-74af9ccc6a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:11.923583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a129d-3838-4b6c-ba3c-74af9ccc6a34 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:11.923867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a129d-3838-4b6c-ba3c-74af9ccc6a34 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:11.924146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a129d-3838-4b6c-ba3c-74af9ccc6a34 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:11.924868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a129d-3838-4b6c-ba3c-74af9ccc6a34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:11.925176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a129d-3838-4b6c-ba3c-74af9ccc6a34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:11.926566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a129d-3838-4b6c-ba3c-74af9ccc6a34 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:11.931025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d014-c968-4f5b-a175-3b0a7f9d8459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:11.932461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d014-c968-4f5b-a175-3b0a7f9d8459 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:11.938869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 535/1068] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:11.944663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a129d-3838-4b6c-ba3c-74af9ccc6a34 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-9599d014-c968-4f5b-a175-3b0a7f9d8459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:36:11.952313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:11.953236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:11.954212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:36:11.954212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:36:11.954212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:11.954212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:11.954212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 534/1069] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:11.959622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d014-c968-4f5b-a175-3b0a7f9d8459 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:11.985615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:12.054938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a129d-3838-4b6c-ba3c-74af9ccc6a34 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:12.076006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d014-c968-4f5b-a175-3b0a7f9d8459 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:12.091735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72338862-80f7-45d6-bf37-b7cc0e175ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:12.092541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72338862-80f7-45d6-bf37-b7cc0e175ec7 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:12.092827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72338862-80f7-45d6-bf37-b7cc0e175ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:12.093103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72338862-80f7-45d6-bf37-b7cc0e175ec7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:12.096431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:12.101555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72338862-80f7-45d6-bf37-b7cc0e175ec7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:12.103027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72338862-80f7-45d6-bf37-b7cc0e175ec7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:12.112208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a3f34-65dd-4af9-837a-5efe604175a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:12.112878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a3f34-65dd-4af9-837a-5efe604175a8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:12.113161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a3f34-65dd-4af9-837a-5efe604175a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:12.113438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a3f34-65dd-4af9-837a-5efe604175a8 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:12.114194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a3f34-65dd-4af9-837a-5efe604175a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:12.114506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a3f34-65dd-4af9-837a-5efe604175a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:12.115872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a3f34-65dd-4af9-837a-5efe604175a8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-72338862-80f7-45d6-bf37-b7cc0e175ec7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:36:12.117307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:12.119043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:12.120010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 536/1070] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:12.124253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72338862-80f7-45d6-bf37-b7cc0e175ec7 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:12.133732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 535/1071] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:12.240469 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72338862-80f7-45d6-bf37-b7cc0e175ec7 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:12.277918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023560b-f0f1-4c74-b456-46aa466da768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:12.278716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023560b-f0f1-4c74-b456-46aa466da768 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:12.278906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023560b-f0f1-4c74-b456-46aa466da768 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:12.279162 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023560b-f0f1-4c74-b456-46aa466da768 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:12.280013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023560b-f0f1-4c74-b456-46aa466da768 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:12.280326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023560b-f0f1-4c74-b456-46aa466da768 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:12.281772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023560b-f0f1-4c74-b456-46aa466da768 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:12.301052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 537/1072] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 14 15:36:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:12.307123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a3f34-65dd-4af9-837a-5efe604175a8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:12.402340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:12.420651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a3f34-65dd-4af9-837a-5efe604175a8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:12.458525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0d528-59f4-443c-b653-527d09df9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:12.459329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0d528-59f4-443c-b653-527d09df9e1e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:12.459613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0d528-59f4-443c-b653-527d09df9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:12.459892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0d528-59f4-443c-b653-527d09df9e1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:12.470519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0d528-59f4-443c-b653-527d09df9e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:12.471972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0d528-59f4-443c-b653-527d09df9e1e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-b9d0d528-59f4-443c-b653-527d09df9e1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:36:12.488867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:12.489960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:12.490868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:36:12.490868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:36:12.490868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:12.490868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:12.490868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 536/1073] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:12.496522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023560b-f0f1-4c74-b456-46aa466da768 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:12.553377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:12.598412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023560b-f0f1-4c74-b456-46aa466da768 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:12.631804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924526c-5fac-48a0-b7eb-25264abc53c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:12.632466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924526c-5fac-48a0-b7eb-25264abc53c0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:12.632751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924526c-5fac-48a0-b7eb-25264abc53c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:12.633028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924526c-5fac-48a0-b7eb-25264abc53c0 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:12.633748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924526c-5fac-48a0-b7eb-25264abc53c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:12.634086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924526c-5fac-48a0-b7eb-25264abc53c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:12.635432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924526c-5fac-48a0-b7eb-25264abc53c0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:12.646926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 538/1074] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:12.653170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0d528-59f4-443c-b653-527d09df9e1e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:12.750422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0d528-59f4-443c-b653-527d09df9e1e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:12.783640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8017b5c-a11e-422d-bc18-08018db7cebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:12.784424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8017b5c-a11e-422d-bc18-08018db7cebd None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:12.784708 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8017b5c-a11e-422d-bc18-08018db7cebd None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:12.784992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8017b5c-a11e-422d-bc18-08018db7cebd None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:12.786903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:12.792056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8017b5c-a11e-422d-bc18-08018db7cebd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:12.793527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8017b5c-a11e-422d-bc18-08018db7cebd None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-d8017b5c-a11e-422d-bc18-08018db7cebd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:36:12.807514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:12.808506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:12.809418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:36:12.809418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:36:12.809418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:36:12.809418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:36:12.809418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:36:12.809418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:36:12.809418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:36:12.809418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:12.809418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 537/1075] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:36:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:12.814709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924526c-5fac-48a0-b7eb-25264abc53c0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:12.913678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924526c-5fac-48a0-b7eb-25264abc53c0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:12.927575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:12.946857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa3558-47e1-4490-bd83-657fa6c28867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:12.947514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa3558-47e1-4490-bd83-657fa6c28867 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:12.947844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa3558-47e1-4490-bd83-657fa6c28867 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:12.948144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa3558-47e1-4490-bd83-657fa6c28867 None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:12.948925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa3558-47e1-4490-bd83-657fa6c28867 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:12.949231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa3558-47e1-4490-bd83-657fa6c28867 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:12.950740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa3558-47e1-4490-bd83-657fa6c28867 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:12.962844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 539/1076] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:36:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:12.969370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8017b5c-a11e-422d-bc18-08018db7cebd None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:13.073626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8017b5c-a11e-422d-bc18-08018db7cebd None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:13.108585 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197267b3-d981-4a2a-836d-9710413b606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:13.109476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197267b3-d981-4a2a-836d-9710413b606d None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:13.109763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197267b3-d981-4a2a-836d-9710413b606d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:13.110070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197267b3-d981-4a2a-836d-9710413b606d None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/ce88c973feef4627a1b871ee9cb26511/roles/5ccecebd4103442484b3dadfbfbf47c4` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:13.112255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197267b3-d981-4a2a-836d-9710413b606d None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=ce88c973feef4627a1b871ee9cb26511, role_id=5ccecebd4103442484b3dadfbfbf47c4)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:13.113689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197267b3-d981-4a2a-836d-9710413b606d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:13.211652 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 538/1077] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:36:12 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/ce88c973feef4627a1b871ee9cb26511/roles/5ccecebd4103442484b3dadfbfbf47c4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:13.441432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed80fe-4b12-45b2-95ae-bd7562f997e0 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:13.441985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed80fe-4b12-45b2-95ae-bd7562f997e0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:13.442073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed80fe-4b12-45b2-95ae-bd7562f997e0 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:13.443563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 540/1078] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:13.448662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebb3ec-00fb-489a-95c7-2612e61d033b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:13.449091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebb3ec-00fb-489a-95c7-2612e61d033b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:13.449227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebb3ec-00fb-489a-95c7-2612e61d033b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:13.465992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d8ebb3ec-00fb-489a-95c7-2612e61d033b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:13.467158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d8ebb3ec-00fb-489a-95c7-2612e61d033b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:13.518224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:13.535612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 539/1079] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:13.558938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed80fe-4b12-45b2-95ae-bd7562f997e0 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:13.660794 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed80fe-4b12-45b2-95ae-bd7562f997e0 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:13.694085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a29348-895f-4e3b-b6e9-7e449d4ba57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:13.694971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a29348-895f-4e3b-b6e9-7e449d4ba57a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:13.695248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a29348-895f-4e3b-b6e9-7e449d4ba57a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:13.695521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a29348-895f-4e3b-b6e9-7e449d4ba57a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:13.706065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a29348-895f-4e3b-b6e9-7e449d4ba57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-e2a29348-895f-4e3b-b6e9-7e449d4ba57a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:13.713536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:13.714591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:13.715892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 541/1080] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:36:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:13.720885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebb3ec-00fb-489a-95c7-2612e61d033b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:13.819536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebb3ec-00fb-489a-95c7-2612e61d033b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:13.853386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850c47c-0398-40e6-ac18-aee20d88df10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:13.854093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850c47c-0398-40e6-ac18-aee20d88df10 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:13.854378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850c47c-0398-40e6-ac18-aee20d88df10 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:13.854653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850c47c-0398-40e6-ac18-aee20d88df10 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:13.855466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850c47c-0398-40e6-ac18-aee20d88df10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:13.855775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850c47c-0398-40e6-ac18-aee20d88df10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:13.857154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850c47c-0398-40e6-ac18-aee20d88df10 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:13.865210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 540/1081] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:36:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:13.871588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a29348-895f-4e3b-b6e9-7e449d4ba57a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:13.906017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:13.971575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a29348-895f-4e3b-b6e9-7e449d4ba57a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:14.003818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ce905-60ae-4fb9-845d-7e551f33994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:14.004693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ce905-60ae-4fb9-845d-7e551f33994d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:14.004979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ce905-60ae-4fb9-845d-7e551f33994d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:14.005257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ce905-60ae-4fb9-845d-7e551f33994d None admin] PATH_INFO: `/v3/users/81717922951641bda7ba774999b98f50` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:14.006456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ce905-60ae-4fb9-845d-7e551f33994d None admin] RBAC: Authorizing `identity:get_user(user_id=81717922951641bda7ba774999b98f50)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:14.007877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ce905-60ae-4fb9-845d-7e551f33994d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:14.009501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 542/1082] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:13 2022] GET /identity/v3/users/81717922951641bda7ba774999b98f50 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:14.015825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850c47c-0398-40e6-ac18-aee20d88df10 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:14.065799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:14.117140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850c47c-0398-40e6-ac18-aee20d88df10 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:14.150446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17db0dfe-2cbf-49d5-a856-b00b52644a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:14.151311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db0dfe-2cbf-49d5-a856-b00b52644a12 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:14.151598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db0dfe-2cbf-49d5-a856-b00b52644a12 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:14.151876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17db0dfe-2cbf-49d5-a856-b00b52644a12 None admin] PATH_INFO: `/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:14.153055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db0dfe-2cbf-49d5-a856-b00b52644a12 None admin] RBAC: Authorizing `identity:get_project(project_id=51cc6bd9b1d34d0d92a6589f9e7be953)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:14.154518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17db0dfe-2cbf-49d5-a856-b00b52644a12 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:14.156237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 541/1083] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:14 2022] GET /identity/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:14.162231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ce905-60ae-4fb9-845d-7e551f33994d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:14.259355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ce905-60ae-4fb9-845d-7e551f33994d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:14.260793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:14.292954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82975dd-bc88-4d56-b647-7179d43d3601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:14.293898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82975dd-bc88-4d56-b647-7179d43d3601 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:14.294180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82975dd-bc88-4d56-b647-7179d43d3601 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:14.294454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82975dd-bc88-4d56-b647-7179d43d3601 None admin] PATH_INFO: `/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953/users/81717922951641bda7ba774999b98f50/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:14.296542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82975dd-bc88-4d56-b647-7179d43d3601 None admin] RBAC: Authorizing `identity:create_grant(project_id=51cc6bd9b1d34d0d92a6589f9e7be953, user_id=81717922951641bda7ba774999b98f50, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:14.298037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82975dd-bc88-4d56-b647-7179d43d3601 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:14.372010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 543/1084] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:36:14 2022] PUT /identity/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953/users/81717922951641bda7ba774999b98f50/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:36:15.211847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d34aba-911a-4686-ab24-69f117c7dff7 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:15.212339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d34aba-911a-4686-ab24-69f117c7dff7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:15.212454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d34aba-911a-4686-ab24-69f117c7dff7 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:15.213781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 542/1085] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:15.219046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390eb00-18ad-4c15-9c0e-33e128dd91e9 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:15.219447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390eb00-18ad-4c15-9c0e-33e128dd91e9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:15.219611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390eb00-18ad-4c15-9c0e-33e128dd91e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:15.236096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-5390eb00-18ad-4c15-9c0e-33e128dd91e9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:15.237300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5390eb00-18ad-4c15-9c0e-33e128dd91e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:15.302484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 544/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:15.327115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d34aba-911a-4686-ab24-69f117c7dff7 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:15.426181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d34aba-911a-4686-ab24-69f117c7dff7 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:15.440617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:15.462106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f41e4-1199-4139-89d9-4cb4a00311b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:15.462921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f41e4-1199-4139-89d9-4cb4a00311b6 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:15.463202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f41e4-1199-4139-89d9-4cb4a00311b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:15.463482 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f41e4-1199-4139-89d9-4cb4a00311b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:15.475357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f41e4-1199-4139-89d9-4cb4a00311b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-ce0f41e4-1199-4139-89d9-4cb4a00311b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:15.484043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:15.485143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:15.486078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 543/1087] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 14 15:36:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:15.492277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390eb00-18ad-4c15-9c0e-33e128dd91e9 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:15.589191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:15.598279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390eb00-18ad-4c15-9c0e-33e128dd91e9 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:15.630663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdb6bd-07ba-4eff-89f5-e95692865257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:15.631329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdb6bd-07ba-4eff-89f5-e95692865257 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:15.631610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdb6bd-07ba-4eff-89f5-e95692865257 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:15.631884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cdb6bd-07ba-4eff-89f5-e95692865257 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:15.632678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdb6bd-07ba-4eff-89f5-e95692865257 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:15.632995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdb6bd-07ba-4eff-89f5-e95692865257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:15.634874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cdb6bd-07ba-4eff-89f5-e95692865257 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:15.643577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 545/1088] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 14 15:36:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:15.649306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f41e4-1199-4139-89d9-4cb4a00311b6 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:15.747523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f41e4-1199-4139-89d9-4cb4a00311b6 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:15.781681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665934e-5891-48c7-b77c-36495793fbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:15.782488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665934e-5891-48c7-b77c-36495793fbc5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:15.782771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665934e-5891-48c7-b77c-36495793fbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:15.783045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665934e-5891-48c7-b77c-36495793fbc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:15.793463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665934e-5891-48c7-b77c-36495793fbc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:15.794939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665934e-5891-48c7-b77c-36495793fbc5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-2665934e-5891-48c7-b77c-36495793fbc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:36:15.812622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:15.813512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:15.814461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:36:15.814461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:36:15.814461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:15.814461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:15.814461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 544/1089] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:15.819680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdb6bd-07ba-4eff-89f5-e95692865257 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:15.918292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cdb6bd-07ba-4eff-89f5-e95692865257 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:15.950953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3c157-1e88-4dd5-9c40-ce1d50a53f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:15.951613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3c157-1e88-4dd5-9c40-ce1d50a53f2e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:15.951898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3c157-1e88-4dd5-9c40-ce1d50a53f2e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:15.952178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3c157-1e88-4dd5-9c40-ce1d50a53f2e None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:15.952902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3c157-1e88-4dd5-9c40-ce1d50a53f2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:15.953212 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3c157-1e88-4dd5-9c40-ce1d50a53f2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:15.954612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3c157-1e88-4dd5-9c40-ce1d50a53f2e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:15.966790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 546/1090] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:15.972471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665934e-5891-48c7-b77c-36495793fbc5 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:16.000512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:16.077765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665934e-5891-48c7-b77c-36495793fbc5 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:16.128846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42be27-3534-483b-9601-a5628f2e04f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:16.129634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42be27-3534-483b-9601-a5628f2e04f4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.129947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42be27-3534-483b-9601-a5628f2e04f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.130226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42be27-3534-483b-9601-a5628f2e04f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.138663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42be27-3534-483b-9601-a5628f2e04f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:16.140101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42be27-3534-483b-9601-a5628f2e04f4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-7a42be27-3534-483b-9601-a5628f2e04f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:36:16.154801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:16.155788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:16.156771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 545/1091] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 14 15:36:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:16.161728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3c157-1e88-4dd5-9c40-ce1d50a53f2e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:16.177916 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:16.259423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3c157-1e88-4dd5-9c40-ce1d50a53f2e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:16.293143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e294c8-4179-4ce6-a02c-d6bce4bb7f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:16.293842 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e294c8-4179-4ce6-a02c-d6bce4bb7f5b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.294127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e294c8-4179-4ce6-a02c-d6bce4bb7f5b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.294406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e294c8-4179-4ce6-a02c-d6bce4bb7f5b None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.295245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e294c8-4179-4ce6-a02c-d6bce4bb7f5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:16.295558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e294c8-4179-4ce6-a02c-d6bce4bb7f5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:16.296984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e294c8-4179-4ce6-a02c-d6bce4bb7f5b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:16.313769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 547/1092] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 14 15:36:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:16.319654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42be27-3534-483b-9601-a5628f2e04f4 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:16.377574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f42204-faaa-418d-a6e6-dd5a5f3e99ce None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.377896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f42204-faaa-418d-a6e6-dd5a5f3e99ce None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.378176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f42204-faaa-418d-a6e6-dd5a5f3e99ce None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.379589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 548/1093] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:36:16.384481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa08de-0abb-4c92-91a4-41031e33f197 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.384774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa08de-0abb-4c92-91a4-41031e33f197 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.385051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa08de-0abb-4c92-91a4-41031e33f197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.402365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-81aa08de-0abb-4c92-91a4-41031e33f197 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:16.403572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-81aa08de-0abb-4c92-91a4-41031e33f197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:16.406963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:16.421681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42be27-3534-483b-9601-a5628f2e04f4 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:16.456459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbabef-9343-4541-b355-43d5f11a7efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:16.457106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbabef-9343-4541-b355-43d5f11a7efb None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.457386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbabef-9343-4541-b355-43d5f11a7efb None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.457660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbabef-9343-4541-b355-43d5f11a7efb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.468704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 549/1094] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 15:36:16.492196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa08de-0abb-4c92-91a4-41031e33f197 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:16.494276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:16.525000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbabef-9343-4541-b355-43d5f11a7efb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:16.526517 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbabef-9343-4541-b355-43d5f11a7efb None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:16.590847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa08de-0abb-4c92-91a4-41031e33f197 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:16.608622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:16.624450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 546/1095] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:16.624741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b234c-9bfc-49b3-ba7a-b140e673ec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:16.625539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b234c-9bfc-49b3-ba7a-b140e673ec47 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.625847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b234c-9bfc-49b3-ba7a-b140e673ec47 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.626127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b234c-9bfc-49b3-ba7a-b140e673ec47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.630249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbabef-9343-4541-b355-43d5f11a7efb None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:16.637113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b234c-9bfc-49b3-ba7a-b140e673ec47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-797b234c-9bfc-49b3-ba7a-b140e673ec47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:16.644630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:16.645796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:16.647103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 550/1096] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:36:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:16.652108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b234c-9bfc-49b3-ba7a-b140e673ec47 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:16.713466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:16.730245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbabef-9343-4541-b355-43d5f11a7efb None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:16.752206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b234c-9bfc-49b3-ba7a-b140e673ec47 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:16.764733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eabec8-4a68-4721-9e62-4a7f44d01359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:16.765523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabec8-4a68-4721-9e62-4a7f44d01359 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.765808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabec8-4a68-4721-9e62-4a7f44d01359 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.766110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabec8-4a68-4721-9e62-4a7f44d01359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.776120 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabec8-4a68-4721-9e62-4a7f44d01359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:16.778137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabec8-4a68-4721-9e62-4a7f44d01359 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:16.785251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbb3e4-41b0-4c5e-b241-0f8d6953756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:16.785936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbb3e4-41b0-4c5e-b241-0f8d6953756e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.786221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbb3e4-41b0-4c5e-b241-0f8d6953756e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.786498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbb3e4-41b0-4c5e-b241-0f8d6953756e None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.787258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbb3e4-41b0-4c5e-b241-0f8d6953756e None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:16.787542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbb3e4-41b0-4c5e-b241-0f8d6953756e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:16.788882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbb3e4-41b0-4c5e-b241-0f8d6953756e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-b6eabec8-4a68-4721-9e62-4a7f44d01359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:36:16.795741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:16.796923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:16.798195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:36:16.798195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:36:16.798195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:36:16.798195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:16.799363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 547/1097] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:36:16.801849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 551/1098] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:36:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:16.802563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eabec8-4a68-4721-9e62-4a7f44d01359 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:16.802665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbb3e4-41b0-4c5e-b241-0f8d6953756e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:16.842474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:16.902150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbb3e4-41b0-4c5e-b241-0f8d6953756e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:16.902449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eabec8-4a68-4721-9e62-4a7f44d01359 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:16.924590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:16.935594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf0e7a-8688-494e-8f0f-45478339fa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:16.936469 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf0e7a-8688-494e-8f0f-45478339fa80 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.936680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a6d7-1fac-4217-8c71-b225675c3a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:16.936808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf0e7a-8688-494e-8f0f-45478339fa80 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.937014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf0e7a-8688-494e-8f0f-45478339fa80 None admin] PATH_INFO: `/v3/users/81717922951641bda7ba774999b98f50` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.937323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a6d7-1fac-4217-8c71-b225675c3a72 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:16.937613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a6d7-1fac-4217-8c71-b225675c3a72 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:16.937923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a6d7-1fac-4217-8c71-b225675c3a72 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:16.938230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf0e7a-8688-494e-8f0f-45478339fa80 None admin] RBAC: Authorizing `identity:get_user(user_id=81717922951641bda7ba774999b98f50)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:16.938650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991a6d7-1fac-4217-8c71-b225675c3a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:16.938961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991a6d7-1fac-4217-8c71-b225675c3a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:16.939656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf0e7a-8688-494e-8f0f-45478339fa80 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:16.940632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991a6d7-1fac-4217-8c71-b225675c3a72 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:16.941561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 552/1099] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:16 2022] GET /identity/v3/users/81717922951641bda7ba774999b98f50 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:16.947181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf0e7a-8688-494e-8f0f-45478339fa80 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:16.957988 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 548/1100] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:16.958966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a6d7-1fac-4217-8c71-b225675c3a72 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:17.047639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf0e7a-8688-494e-8f0f-45478339fa80 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:17.060184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:17.064237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a6d7-1fac-4217-8c71-b225675c3a72 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:17.082311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677012e1-bc7c-4f6b-bdbf-bdbc15bcfcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:17.082965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677012e1-bc7c-4f6b-bdbf-bdbc15bcfcbf None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:17.083250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677012e1-bc7c-4f6b-bdbf-bdbc15bcfcbf None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:17.083528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677012e1-bc7c-4f6b-bdbf-bdbc15bcfcbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:17.098709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb1619-dc5a-4053-8706-8fef22367fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:17.099501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb1619-dc5a-4053-8706-8fef22367fd6 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:17.099788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb1619-dc5a-4053-8706-8fef22367fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:17.100068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb1619-dc5a-4053-8706-8fef22367fd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:17.107420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb1619-dc5a-4053-8706-8fef22367fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:17.108872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb1619-dc5a-4053-8706-8fef22367fd6 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-cffb1619-dc5a-4053-8706-8fef22367fd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:36:17.123113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:17.124039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:17.124948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:36:17.124948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:36:17.124948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:36:17.124948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:36:17.124948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:36:17.124948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:36:17.124948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:36:17.124948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:17.124948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 549/1101] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:36:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:17.130163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb1619-dc5a-4053-8706-8fef22367fd6 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:17.142028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:17.151389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677012e1-bc7c-4f6b-bdbf-bdbc15bcfcbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:17.152898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677012e1-bc7c-4f6b-bdbf-bdbc15bcfcbf None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:17.228466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb1619-dc5a-4053-8706-8fef22367fd6 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:17.252142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 553/1102] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:17.257942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677012e1-bc7c-4f6b-bdbf-bdbc15bcfcbf None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:17.262477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10ec08-9b44-4304-914f-d4e8cf753dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:17.263140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10ec08-9b44-4304-914f-d4e8cf753dc8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:17.263424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10ec08-9b44-4304-914f-d4e8cf753dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:17.263705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10ec08-9b44-4304-914f-d4e8cf753dc8 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:17.264489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10ec08-9b44-4304-914f-d4e8cf753dc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:17.264799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10ec08-9b44-4304-914f-d4e8cf753dc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:17.266264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10ec08-9b44-4304-914f-d4e8cf753dc8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:17.278000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 550/1103] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:36:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:17.283976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10ec08-9b44-4304-914f-d4e8cf753dc8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:17.302901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:17.348933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:17.357336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677012e1-bc7c-4f6b-bdbf-bdbc15bcfcbf None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:17.384003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10ec08-9b44-4304-914f-d4e8cf753dc8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:17.391244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fba4b-5c36-4d28-8e0c-f590ce46e251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:17.392174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fba4b-5c36-4d28-8e0c-f590ce46e251 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:17.392463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fba4b-5c36-4d28-8e0c-f590ce46e251 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:17.392738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574fba4b-5c36-4d28-8e0c-f590ce46e251 None admin] PATH_INFO: `/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:17.393956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574fba4b-5c36-4d28-8e0c-f590ce46e251 None admin] RBAC: Authorizing `identity:get_project(project_id=51cc6bd9b1d34d0d92a6589f9e7be953)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:17.395373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574fba4b-5c36-4d28-8e0c-f590ce46e251 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:17.397367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 554/1104] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:17 2022] GET /identity/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:17.403537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fba4b-5c36-4d28-8e0c-f590ce46e251 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:17.420288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab10efd-1274-4ecf-acb6-d06b8e40e672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:17.420943 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab10efd-1274-4ecf-acb6-d06b8e40e672 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:17.421227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab10efd-1274-4ecf-acb6-d06b8e40e672 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:17.421512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab10efd-1274-4ecf-acb6-d06b8e40e672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:17.422358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab10efd-1274-4ecf-acb6-d06b8e40e672 None admin] RBAC: Adding query filter params (role.id=5ccecebd4103442484b3dadfbfbf47c4, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=ce88c973feef4627a1b871ee9cb26511) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:17.422675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab10efd-1274-4ecf-acb6-d06b8e40e672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:17.424099 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab10efd-1274-4ecf-acb6-d06b8e40e672 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:17.430070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 551/1105] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:36:17 2022] GET /identity/v3/role_assignments?user.id=ce88c973feef4627a1b871ee9cb26511&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=5ccecebd4103442484b3dadfbfbf47c4 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:17.509644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574fba4b-5c36-4d28-8e0c-f590ce46e251 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:17.547056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6435d-8a63-485e-ae90-64db2a4fc9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:17.547735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6435d-8a63-485e-ae90-64db2a4fc9a2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:17.548019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6435d-8a63-485e-ae90-64db2a4fc9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:17.548297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6435d-8a63-485e-ae90-64db2a4fc9a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:17.549144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6435d-8a63-485e-ae90-64db2a4fc9a2 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=51cc6bd9b1d34d0d92a6589f9e7be953, user.id=81717922951641bda7ba774999b98f50) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:17.549462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6435d-8a63-485e-ae90-64db2a4fc9a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:17.550957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6435d-8a63-485e-ae90-64db2a4fc9a2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:17.557645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 555/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:36:17 2022] GET /identity/v3/role_assignments?user.id=81717922951641bda7ba774999b98f50&scope.project.id=51cc6bd9b1d34d0d92a6589f9e7be953&role.id=abfb60dccca7470295258595a66a5d17 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:36:19.603742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c18751-bbed-4178-86d3-5f1f8abd0cae None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:19.604295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c18751-bbed-4178-86d3-5f1f8abd0cae None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:19.604295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c18751-bbed-4178-86d3-5f1f8abd0cae None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:19.606314 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 552/1107] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:19.611652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f99149-4893-4453-9de1-dbc032a1754c None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:19.612059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f99149-4893-4453-9de1-dbc032a1754c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:19.612207 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f99149-4893-4453-9de1-dbc032a1754c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:19.629053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a9f99149-4893-4453-9de1-dbc032a1754c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:19.630267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a9f99149-4893-4453-9de1-dbc032a1754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:19.646965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:19.696274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 556/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:19.720768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c18751-bbed-4178-86d3-5f1f8abd0cae None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:19.802612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:19.821478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c18751-bbed-4178-86d3-5f1f8abd0cae None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:19.855042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a03a03-db0c-48af-a421-1bc7732dd2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:19.855764 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03a03-db0c-48af-a421-1bc7732dd2c4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:19.856050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03a03-db0c-48af-a421-1bc7732dd2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:19.856323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03a03-db0c-48af-a421-1bc7732dd2c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:19.857489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a03a03-db0c-48af-a421-1bc7732dd2c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:19.858963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a03a03-db0c-48af-a421-1bc7732dd2c4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:19.860570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 553/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:19.866272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f99149-4893-4453-9de1-dbc032a1754c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:19.962470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f99149-4893-4453-9de1-dbc032a1754c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:19.994866 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c097b37-db3c-4507-9528-5be2612367c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:19.996028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c097b37-db3c-4507-9528-5be2612367c9 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:19.996310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c097b37-db3c-4507-9528-5be2612367c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:19.996586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c097b37-db3c-4507-9528-5be2612367c9 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:19.997286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c097b37-db3c-4507-9528-5be2612367c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:19.998780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c097b37-db3c-4507-9528-5be2612367c9 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:20.123585 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 557/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:36:19 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:22.208375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f0a5a-9006-4ad7-97f5-d3ce0a2e88f1 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:22.208953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f0a5a-9006-4ad7-97f5-d3ce0a2e88f1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:22.208953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f0a5a-9006-4ad7-97f5-d3ce0a2e88f1 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:22.210329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 554/1111] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:22.215592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec36f8e-b6f0-46cd-8a12-0beb9239ce71 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:22.215999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec36f8e-b6f0-46cd-8a12-0beb9239ce71 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:22.216150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec36f8e-b6f0-46cd-8a12-0beb9239ce71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:22.232727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-bec36f8e-b6f0-46cd-8a12-0beb9239ce71 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:22.233957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-bec36f8e-b6f0-46cd-8a12-0beb9239ce71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:22.236642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:22.300238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 558/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:22.324095 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f0a5a-9006-4ad7-97f5-d3ce0a2e88f1 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:22.423950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f0a5a-9006-4ad7-97f5-d3ce0a2e88f1 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:22.457537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493170d8-aa03-48e0-8d1b-df1eb4b5b55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:22.458403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493170d8-aa03-48e0-8d1b-df1eb4b5b55a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:22.458685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493170d8-aa03-48e0-8d1b-df1eb4b5b55a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:22.458966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493170d8-aa03-48e0-8d1b-df1eb4b5b55a None admin] PATH_INFO: `/v3/roles/2806b70c57be44f9b260ff5322bf02fb` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:22.460419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493170d8-aa03-48e0-8d1b-df1eb4b5b55a None admin] RBAC: Authorizing `identity:get_role(role_id=2806b70c57be44f9b260ff5322bf02fb)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:22.461781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493170d8-aa03-48e0-8d1b-df1eb4b5b55a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:22.463150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 555/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:22 2022] GET /identity/v3/roles/2806b70c57be44f9b260ff5322bf02fb => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:22.468752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec36f8e-b6f0-46cd-8a12-0beb9239ce71 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:22.566002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec36f8e-b6f0-46cd-8a12-0beb9239ce71 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:22.598203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c26435-4b41-4169-a7ec-c286ff35e24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:22.599072 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c26435-4b41-4169-a7ec-c286ff35e24b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:22.599359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c26435-4b41-4169-a7ec-c286ff35e24b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:22.599641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c26435-4b41-4169-a7ec-c286ff35e24b None admin] PATH_INFO: `/v3/users/5752aab3761a453db017560d6e7fcf62` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:22.610114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:22.613297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c26435-4b41-4169-a7ec-c286ff35e24b None admin] RBAC: Authorizing `identity:get_user(user_id=5752aab3761a453db017560d6e7fcf62)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:22.614763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c26435-4b41-4169-a7ec-c286ff35e24b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:22.616411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 559/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:22 2022] GET /identity/v3/users/5752aab3761a453db017560d6e7fcf62 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:22.622758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493170d8-aa03-48e0-8d1b-df1eb4b5b55a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:22.641424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:22.721105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493170d8-aa03-48e0-8d1b-df1eb4b5b55a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:22.755391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616165d-d191-409b-8251-e7cecad05280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:22.756035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616165d-d191-409b-8251-e7cecad05280 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:22.756322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616165d-d191-409b-8251-e7cecad05280 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:22.756599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616165d-d191-409b-8251-e7cecad05280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:22.825370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616165d-d191-409b-8251-e7cecad05280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:22.826785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616165d-d191-409b-8251-e7cecad05280 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:22.844280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:22.923701 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 556/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:22.929660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c26435-4b41-4169-a7ec-c286ff35e24b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:23.025831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c26435-4b41-4169-a7ec-c286ff35e24b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:23.060164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2f164-3dea-4dc7-97e7-dd6d9ef77d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:23.061029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2f164-3dea-4dc7-97e7-dd6d9ef77d66 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:23.061313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2f164-3dea-4dc7-97e7-dd6d9ef77d66 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:23.061589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2f164-3dea-4dc7-97e7-dd6d9ef77d66 None admin] PATH_INFO: `/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:23.062808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2f164-3dea-4dc7-97e7-dd6d9ef77d66 None admin] RBAC: Authorizing `identity:get_project(project_id=51cc6bd9b1d34d0d92a6589f9e7be953)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:23.064269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2f164-3dea-4dc7-97e7-dd6d9ef77d66 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:23.066266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 560/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:22 2022] GET /identity/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:23.072264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616165d-d191-409b-8251-e7cecad05280 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:23.169735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616165d-d191-409b-8251-e7cecad05280 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:23.186485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:23.203469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f36053-51f2-48ed-904e-540f3bfa30d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:23.207176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f36053-51f2-48ed-904e-540f3bfa30d4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:23.207176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f36053-51f2-48ed-904e-540f3bfa30d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:23.207176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f36053-51f2-48ed-904e-540f3bfa30d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:23.207176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f36053-51f2-48ed-904e-540f3bfa30d4 None admin] RBAC: Adding query filter params (role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=51cc6bd9b1d34d0d92a6589f9e7be953, user.id=5752aab3761a453db017560d6e7fcf62) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:23.207176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f36053-51f2-48ed-904e-540f3bfa30d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:23.207614 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f36053-51f2-48ed-904e-540f3bfa30d4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:23.212519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 557/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:36:23 2022] GET /identity/v3/role_assignments?user.id=5752aab3761a453db017560d6e7fcf62&scope.project.id=51cc6bd9b1d34d0d92a6589f9e7be953&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:25.218618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bae86-5848-404a-b132-1d763147caa9 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.219153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bae86-5848-404a-b132-1d763147caa9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.219153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bae86-5848-404a-b132-1d763147caa9 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.220541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 561/1118] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:25.225863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68030b-979e-4e91-8ae6-48e0a6ccec26 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.226263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68030b-979e-4e91-8ae6-48e0a6ccec26 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.226434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68030b-979e-4e91-8ae6-48e0a6ccec26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.243379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-db68030b-979e-4e91-8ae6-48e0a6ccec26 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:25.244558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-db68030b-979e-4e91-8ae6-48e0a6ccec26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:25.311255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 558/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:25.333884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bae86-5848-404a-b132-1d763147caa9 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.399095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:25.433671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bae86-5848-404a-b132-1d763147caa9 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.466770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da235a4-d4dd-43ed-a7d3-ddc18ae17415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.467619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da235a4-d4dd-43ed-a7d3-ddc18ae17415 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.467901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da235a4-d4dd-43ed-a7d3-ddc18ae17415 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.468176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da235a4-d4dd-43ed-a7d3-ddc18ae17415 None admin] PATH_INFO: `/v3/roles/2806b70c57be44f9b260ff5322bf02fb` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.469624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da235a4-d4dd-43ed-a7d3-ddc18ae17415 None admin] RBAC: Authorizing `identity:get_role(role_id=2806b70c57be44f9b260ff5322bf02fb)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.471009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da235a4-d4dd-43ed-a7d3-ddc18ae17415 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.472355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 562/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/roles/2806b70c57be44f9b260ff5322bf02fb => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:25.478231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68030b-979e-4e91-8ae6-48e0a6ccec26 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.574684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68030b-979e-4e91-8ae6-48e0a6ccec26 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.587576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:25.609350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588fb32-b70d-49b9-bc4b-7e69d995af5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.610247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588fb32-b70d-49b9-bc4b-7e69d995af5c None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.610529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588fb32-b70d-49b9-bc4b-7e69d995af5c None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.610802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588fb32-b70d-49b9-bc4b-7e69d995af5c None admin] PATH_INFO: `/v3/users/5752aab3761a453db017560d6e7fcf62` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.611973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588fb32-b70d-49b9-bc4b-7e69d995af5c None admin] RBAC: Authorizing `identity:get_user(user_id=5752aab3761a453db017560d6e7fcf62)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.613409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588fb32-b70d-49b9-bc4b-7e69d995af5c None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.615082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 559/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/users/5752aab3761a453db017560d6e7fcf62 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:25.621515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da235a4-d4dd-43ed-a7d3-ddc18ae17415 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.717176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da235a4-d4dd-43ed-a7d3-ddc18ae17415 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.749566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2dfe2b-6015-4c63-bb7f-ec48cb310d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.750433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dfe2b-6015-4c63-bb7f-ec48cb310d35 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.750712 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dfe2b-6015-4c63-bb7f-ec48cb310d35 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.750982 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dfe2b-6015-4c63-bb7f-ec48cb310d35 None admin] PATH_INFO: `/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.752133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2dfe2b-6015-4c63-bb7f-ec48cb310d35 None admin] RBAC: Authorizing `identity:get_project(project_id=51cc6bd9b1d34d0d92a6589f9e7be953)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.753545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2dfe2b-6015-4c63-bb7f-ec48cb310d35 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.755246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 563/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:25 2022] GET /identity/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:25.761666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588fb32-b70d-49b9-bc4b-7e69d995af5c None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:25.858796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588fb32-b70d-49b9-bc4b-7e69d995af5c None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:25.892247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef4968-0e26-4155-9b69-cb181f73b8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:25.893125 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef4968-0e26-4155-9b69-cb181f73b8f8 None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:25.893407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef4968-0e26-4155-9b69-cb181f73b8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:25.893679 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef4968-0e26-4155-9b69-cb181f73b8f8 None admin] PATH_INFO: `/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953/users/5752aab3761a453db017560d6e7fcf62/roles/2806b70c57be44f9b260ff5322bf02fb` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:25.895875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef4968-0e26-4155-9b69-cb181f73b8f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=51cc6bd9b1d34d0d92a6589f9e7be953, user_id=5752aab3761a453db017560d6e7fcf62, role_id=2806b70c57be44f9b260ff5322bf02fb)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:25.897300 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef4968-0e26-4155-9b69-cb181f73b8f8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:25.967757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 560/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:36:25 2022] PUT /identity/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953/users/5752aab3761a453db017560d6e7fcf62/roles/2806b70c57be44f9b260ff5322bf02fb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:27.940260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128ca2d-3e62-476f-ab32-4882d0b95bb3 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.940796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128ca2d-3e62-476f-ab32-4882d0b95bb3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.940796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128ca2d-3e62-476f-ab32-4882d0b95bb3 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.942229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 564/1124] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:27.947253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68233d-811c-4c8e-9a97-7372d81cf0ad None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:27.947673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68233d-811c-4c8e-9a97-7372d81cf0ad None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:27.947806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68233d-811c-4c8e-9a97-7372d81cf0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:27.965268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ab68233d-811c-4c8e-9a97-7372d81cf0ad None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:27.966476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ab68233d-811c-4c8e-9a97-7372d81cf0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:28.000682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:28.032644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 561/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:28.055963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128ca2d-3e62-476f-ab32-4882d0b95bb3 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.059424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:28.156626 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128ca2d-3e62-476f-ab32-4882d0b95bb3 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.189169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d66d1-afba-4300-8c21-ff1e1368c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.190344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d66d1-afba-4300-8c21-ff1e1368c3fb None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.190570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d66d1-afba-4300-8c21-ff1e1368c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.190841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d66d1-afba-4300-8c21-ff1e1368c3fb None admin] PATH_INFO: `/v3/roles/2806b70c57be44f9b260ff5322bf02fb` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.192285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d66d1-afba-4300-8c21-ff1e1368c3fb None admin] RBAC: Authorizing `identity:get_role(role_id=2806b70c57be44f9b260ff5322bf02fb)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.193740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d66d1-afba-4300-8c21-ff1e1368c3fb None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.195124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 565/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:28 2022] GET /identity/v3/roles/2806b70c57be44f9b260ff5322bf02fb => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:28.201447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68233d-811c-4c8e-9a97-7372d81cf0ad None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.302024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab68233d-811c-4c8e-9a97-7372d81cf0ad None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.334887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db544d8-6f6a-4f2d-a177-e30927f7cc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.335741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db544d8-6f6a-4f2d-a177-e30927f7cc1a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.336021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db544d8-6f6a-4f2d-a177-e30927f7cc1a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.336297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db544d8-6f6a-4f2d-a177-e30927f7cc1a None admin] PATH_INFO: `/v3/users/5752aab3761a453db017560d6e7fcf62` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.337457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db544d8-6f6a-4f2d-a177-e30927f7cc1a None admin] RBAC: Authorizing `identity:get_user(user_id=5752aab3761a453db017560d6e7fcf62)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.338942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db544d8-6f6a-4f2d-a177-e30927f7cc1a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.340618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 562/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:28 2022] GET /identity/v3/users/5752aab3761a453db017560d6e7fcf62 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:28.346456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d66d1-afba-4300-8c21-ff1e1368c3fb None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.414056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:28.444404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d66d1-afba-4300-8c21-ff1e1368c3fb None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.476918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9c8bb-f271-45e4-8bfd-95f734e4fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.477562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9c8bb-f271-45e4-8bfd-95f734e4fd68 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.477871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9c8bb-f271-45e4-8bfd-95f734e4fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.478154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec9c8bb-f271-45e4-8bfd-95f734e4fd68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.543421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9c8bb-f271-45e4-8bfd-95f734e4fd68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.544858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec9c8bb-f271-45e4-8bfd-95f734e4fd68 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.609682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:28.640219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 566/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:28.646296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db544d8-6f6a-4f2d-a177-e30927f7cc1a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.686189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:28.744045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db544d8-6f6a-4f2d-a177-e30927f7cc1a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.777631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab705a-fa5c-4934-b037-5bc8fa97b8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.778504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab705a-fa5c-4934-b037-5bc8fa97b8b8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.778789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab705a-fa5c-4934-b037-5bc8fa97b8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.779065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab705a-fa5c-4934-b037-5bc8fa97b8b8 None admin] PATH_INFO: `/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.780239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab705a-fa5c-4934-b037-5bc8fa97b8b8 None admin] RBAC: Authorizing `identity:get_project(project_id=51cc6bd9b1d34d0d92a6589f9e7be953)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.781743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab705a-fa5c-4934-b037-5bc8fa97b8b8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.783452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 563/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:28 2022] GET /identity/v3/projects/51cc6bd9b1d34d0d92a6589f9e7be953 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:28.790122 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9c8bb-f271-45e4-8bfd-95f734e4fd68 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:28.886948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec9c8bb-f271-45e4-8bfd-95f734e4fd68 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:28.919884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1b8b5-1f20-47f7-a43c-930b659530ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:28.920531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1b8b5-1f20-47f7-a43c-930b659530ee None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:28.920816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1b8b5-1f20-47f7-a43c-930b659530ee None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:28.921090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1b8b5-1f20-47f7-a43c-930b659530ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:28.921949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1b8b5-1f20-47f7-a43c-930b659530ee None admin] RBAC: Adding query filter params (role.id=2806b70c57be44f9b260ff5322bf02fb, scope.project.id=51cc6bd9b1d34d0d92a6589f9e7be953, user.id=5752aab3761a453db017560d6e7fcf62) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:28.922262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1b8b5-1f20-47f7-a43c-930b659530ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:28.923677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1b8b5-1f20-47f7-a43c-930b659530ee None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:28.929364 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 567/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:36:28 2022] GET /identity/v3/role_assignments?user.id=5752aab3761a453db017560d6e7fcf62&scope.project.id=51cc6bd9b1d34d0d92a6589f9e7be953&role.id=2806b70c57be44f9b260ff5322bf02fb => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:30.946870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997dd19-5d1d-40c4-a369-02558f39b042 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.947430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997dd19-5d1d-40c4-a369-02558f39b042 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.947430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997dd19-5d1d-40c4-a369-02558f39b042 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.948878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 564/1131] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:30.953911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a9183-fa2b-407e-be11-9a220c86e866 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:30.954334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a9183-fa2b-407e-be11-9a220c86e866 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:30.954463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a9183-fa2b-407e-be11-9a220c86e866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:30.971363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-611a9183-fa2b-407e-be11-9a220c86e866 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:30.972533 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-611a9183-fa2b-407e-be11-9a220c86e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:30.998625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:31.039822 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 568/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:31.061414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997dd19-5d1d-40c4-a369-02558f39b042 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:31.160806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997dd19-5d1d-40c4-a369-02558f39b042 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:31.167865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:31.194629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5fb56-bf4e-41be-a26b-54f2e2e848d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:31.195414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5fb56-bf4e-41be-a26b-54f2e2e848d8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:31.195696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5fb56-bf4e-41be-a26b-54f2e2e848d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:31.195967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5fb56-bf4e-41be-a26b-54f2e2e848d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:31.197185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5fb56-bf4e-41be-a26b-54f2e2e848d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:31.199243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5fb56-bf4e-41be-a26b-54f2e2e848d8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:31.201533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 565/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:31.207231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a9183-fa2b-407e-be11-9a220c86e866 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:31.302703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a9183-fa2b-407e-be11-9a220c86e866 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:31.335556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45eee7-6bc6-453e-8811-ac9c6646d7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:31.336210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45eee7-6bc6-453e-8811-ac9c6646d7dd None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:31.336493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45eee7-6bc6-453e-8811-ac9c6646d7dd None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:31.336769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45eee7-6bc6-453e-8811-ac9c6646d7dd None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:31.337485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45eee7-6bc6-453e-8811-ac9c6646d7dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:31.338991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45eee7-6bc6-453e-8811-ac9c6646d7dd None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:31.457197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 569/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:36:31 2022] POST /identity/v3/projects => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:33.487478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914f47-f6f4-4552-aefe-2e045f8d6a79 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:33.488022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914f47-f6f4-4552-aefe-2e045f8d6a79 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:33.488022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e914f47-f6f4-4552-aefe-2e045f8d6a79 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:33.489388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 566/1135] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:33.494653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2333a0-8d01-4a59-9df2-10887a6fa68e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:33.495070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2333a0-8d01-4a59-9df2-10887a6fa68e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:33.495211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2333a0-8d01-4a59-9df2-10887a6fa68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:33.511793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-9d2333a0-8d01-4a59-9df2-10887a6fa68e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:33.512985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9d2333a0-8d01-4a59-9df2-10887a6fa68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:33.523420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:33.579053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 570/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:33.603962 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e914f47-f6f4-4552-aefe-2e045f8d6a79 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:33.703694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e914f47-f6f4-4552-aefe-2e045f8d6a79 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:33.736680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c4c76-2548-46cd-9aee-eaa82b13fb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:33.737458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c4c76-2548-46cd-9aee-eaa82b13fb54 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:33.737795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c4c76-2548-46cd-9aee-eaa82b13fb54 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:33.738052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c4c76-2548-46cd-9aee-eaa82b13fb54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:33.739218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c4c76-2548-46cd-9aee-eaa82b13fb54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:33.740651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c4c76-2548-46cd-9aee-eaa82b13fb54 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:33.742294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 567/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:36:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:33.748280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2333a0-8d01-4a59-9df2-10887a6fa68e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:33.845332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2333a0-8d01-4a59-9df2-10887a6fa68e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:33.878817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d1599-7eaa-418d-bd54-bd96b790ead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:33.879479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d1599-7eaa-418d-bd54-bd96b790ead8 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:33.879769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d1599-7eaa-418d-bd54-bd96b790ead8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:33.880046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d1599-7eaa-418d-bd54-bd96b790ead8 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:33.880764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d1599-7eaa-418d-bd54-bd96b790ead8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:33.882209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d1599-7eaa-418d-bd54-bd96b790ead8 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:33.982377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 571/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:36:33 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:36.017103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550c8c-619e-4c36-893e-1c6c4eacfa71 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.017655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550c8c-619e-4c36-893e-1c6c4eacfa71 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.017655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550c8c-619e-4c36-893e-1c6c4eacfa71 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.019099 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 568/1139] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:36.024272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c99e2-b10b-4101-ab0c-f5d70d073344 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.024685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c99e2-b10b-4101-ab0c-f5d70d073344 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.024831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c99e2-b10b-4101-ab0c-f5d70d073344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.030422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:36.042533 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c82c99e2-b10b-4101-ab0c-f5d70d073344 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:36.045280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c82c99e2-b10b-4101-ab0c-f5d70d073344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:36.109889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 572/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:36.132619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89550c8c-619e-4c36-893e-1c6c4eacfa71 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.177036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:36.233463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89550c8c-619e-4c36-893e-1c6c4eacfa71 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.266933 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357b4fa-086b-43f6-b7ca-4411d5a348da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.267769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357b4fa-086b-43f6-b7ca-4411d5a348da None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.268002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357b4fa-086b-43f6-b7ca-4411d5a348da None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.268279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357b4fa-086b-43f6-b7ca-4411d5a348da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.278896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0357b4fa-086b-43f6-b7ca-4411d5a348da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-0357b4fa-086b-43f6-b7ca-4411d5a348da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:36.286505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:36.287678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:36.288778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 569/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:36:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:36.294151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c99e2-b10b-4101-ab0c-f5d70d073344 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.390039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c99e2-b10b-4101-ab0c-f5d70d073344 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.418954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:36.423626 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cc983-874f-4b22-b932-7c7ef9cc2af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.424294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cc983-874f-4b22-b932-7c7ef9cc2af1 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.424574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cc983-874f-4b22-b932-7c7ef9cc2af1 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.424847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cc983-874f-4b22-b932-7c7ef9cc2af1 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.425591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cc983-874f-4b22-b932-7c7ef9cc2af1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:36.425916 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cc983-874f-4b22-b932-7c7ef9cc2af1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.427264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cc983-874f-4b22-b932-7c7ef9cc2af1 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:36.435553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 573/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:36:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:36.441606 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357b4fa-086b-43f6-b7ca-4411d5a348da None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.540346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357b4fa-086b-43f6-b7ca-4411d5a348da None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.541788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:36.574858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9130ada-4a88-46f0-adbe-add169595175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.575716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9130ada-4a88-46f0-adbe-add169595175 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.576001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9130ada-4a88-46f0-adbe-add169595175 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.576273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9130ada-4a88-46f0-adbe-add169595175 None admin] PATH_INFO: `/v3/users/3f2d9d25e4484ebd88a24dcdbeca02a1` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.589727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9130ada-4a88-46f0-adbe-add169595175 None admin] RBAC: Authorizing `identity:get_user(user_id=3f2d9d25e4484ebd88a24dcdbeca02a1)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.591192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9130ada-4a88-46f0-adbe-add169595175 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:36.592838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 570/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:36 2022] GET /identity/v3/users/3f2d9d25e4484ebd88a24dcdbeca02a1 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:36.598636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cc983-874f-4b22-b932-7c7ef9cc2af1 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.695675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cc983-874f-4b22-b932-7c7ef9cc2af1 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:36.727706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ec628-ab98-4b5b-9e3c-20a113c7d70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:36.728351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ec628-ab98-4b5b-9e3c-20a113c7d70b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:36.728632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ec628-ab98-4b5b-9e3c-20a113c7d70b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:36.728904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ec628-ab98-4b5b-9e3c-20a113c7d70b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:36.787843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:36.796107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ec628-ab98-4b5b-9e3c-20a113c7d70b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:36.796435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ec628-ab98-4b5b-9e3c-20a113c7d70b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:36.891178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 574/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:36.897290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9130ada-4a88-46f0-adbe-add169595175 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:36.994433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9130ada-4a88-46f0-adbe-add169595175 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:37.027762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb43b-da0e-4865-93bd-d10e720f14b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:37.028609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb43b-da0e-4865-93bd-d10e720f14b0 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:37.028890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb43b-da0e-4865-93bd-d10e720f14b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:37.029164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb43b-da0e-4865-93bd-d10e720f14b0 None admin] PATH_INFO: `/v3/projects/a5677da7953148e5ad8ffb1bff5896ee` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:37.030378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdb43b-da0e-4865-93bd-d10e720f14b0 None admin] RBAC: Authorizing `identity:get_project(project_id=a5677da7953148e5ad8ffb1bff5896ee)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:37.031806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdb43b-da0e-4865-93bd-d10e720f14b0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:37.033492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 571/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:36 2022] GET /identity/v3/projects/a5677da7953148e5ad8ffb1bff5896ee => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:37.039752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ec628-ab98-4b5b-9e3c-20a113c7d70b None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:37.130111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:37.137015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ec628-ab98-4b5b-9e3c-20a113c7d70b None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:37.169646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb663b4b-0ccf-4a46-8265-7e2f4b5ec8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:37.170333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663b4b-0ccf-4a46-8265-7e2f4b5ec8b4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:37.170619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663b4b-0ccf-4a46-8265-7e2f4b5ec8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:37.170897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb663b4b-0ccf-4a46-8265-7e2f4b5ec8b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:37.171721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb663b4b-0ccf-4a46-8265-7e2f4b5ec8b4 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=a5677da7953148e5ad8ffb1bff5896ee, user.id=3f2d9d25e4484ebd88a24dcdbeca02a1) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:37.172033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb663b4b-0ccf-4a46-8265-7e2f4b5ec8b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:37.173458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb663b4b-0ccf-4a46-8265-7e2f4b5ec8b4 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:37.178819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 575/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:36:37 2022] GET /identity/v3/role_assignments?user.id=3f2d9d25e4484ebd88a24dcdbeca02a1&scope.project.id=a5677da7953148e5ad8ffb1bff5896ee&role.id=abfb60dccca7470295258595a66a5d17 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:39.164221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c701184-6d46-436d-b70b-f4988f4faec6 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.164747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c701184-6d46-436d-b70b-f4988f4faec6 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.164747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c701184-6d46-436d-b70b-f4988f4faec6 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.166170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 572/1147] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:39.171317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d33dd-3f5c-4114-a541-8767e562adb4 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.171690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d33dd-3f5c-4114-a541-8767e562adb4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.171882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d33dd-3f5c-4114-a541-8767e562adb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.188721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-335d33dd-3f5c-4114-a541-8767e562adb4 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:39.190204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-335d33dd-3f5c-4114-a541-8767e562adb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:39.254508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 576/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:39.276633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c701184-6d46-436d-b70b-f4988f4faec6 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:39.303834 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:39.377681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c701184-6d46-436d-b70b-f4988f4faec6 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:39.411369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3552884-5f9a-4723-bc15-535563b079e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:39.412150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3552884-5f9a-4723-bc15-535563b079e4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.412432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3552884-5f9a-4723-bc15-535563b079e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.412705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3552884-5f9a-4723-bc15-535563b079e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.423440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3552884-5f9a-4723-bc15-535563b079e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-c3552884-5f9a-4723-bc15-535563b079e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:39.431043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:39.432163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:39.433407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 573/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:36:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:39.438383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d33dd-3f5c-4114-a541-8767e562adb4 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:39.522546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:39.534462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d33dd-3f5c-4114-a541-8767e562adb4 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:39.567940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c58fad-2d7f-4fd1-8663-c390293e6bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:39.568545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c58fad-2d7f-4fd1-8663-c390293e6bb2 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.568832 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c58fad-2d7f-4fd1-8663-c390293e6bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.569110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c58fad-2d7f-4fd1-8663-c390293e6bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.569878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c58fad-2d7f-4fd1-8663-c390293e6bb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:39.570191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c58fad-2d7f-4fd1-8663-c390293e6bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:39.571547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c58fad-2d7f-4fd1-8663-c390293e6bb2 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:39.579938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 577/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:36:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:39.585628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3552884-5f9a-4723-bc15-535563b079e4 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:39.670735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:39.682134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3552884-5f9a-4723-bc15-535563b079e4 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:39.715890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65671dfb-37ca-4f49-9a23-207f2012669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:39.716756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65671dfb-37ca-4f49-9a23-207f2012669a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.717043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65671dfb-37ca-4f49-9a23-207f2012669a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.717320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65671dfb-37ca-4f49-9a23-207f2012669a None admin] PATH_INFO: `/v3/users/3f2d9d25e4484ebd88a24dcdbeca02a1` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.718517 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65671dfb-37ca-4f49-9a23-207f2012669a None admin] RBAC: Authorizing `identity:get_user(user_id=3f2d9d25e4484ebd88a24dcdbeca02a1)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:39.719948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65671dfb-37ca-4f49-9a23-207f2012669a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:39.721561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 574/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:39 2022] GET /identity/v3/users/3f2d9d25e4484ebd88a24dcdbeca02a1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:39.727528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c58fad-2d7f-4fd1-8663-c390293e6bb2 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:39.828126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c58fad-2d7f-4fd1-8663-c390293e6bb2 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:39.860582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bcc575-dd5c-4e57-a9f1-c43423026263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:39.861438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcc575-dd5c-4e57-a9f1-c43423026263 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:39.861724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcc575-dd5c-4e57-a9f1-c43423026263 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:39.862034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcc575-dd5c-4e57-a9f1-c43423026263 None admin] PATH_INFO: `/v3/projects/a5677da7953148e5ad8ffb1bff5896ee` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:39.863215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcc575-dd5c-4e57-a9f1-c43423026263 None admin] RBAC: Authorizing `identity:get_project(project_id=a5677da7953148e5ad8ffb1bff5896ee)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:39.864655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bcc575-dd5c-4e57-a9f1-c43423026263 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:39.866375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 578/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:39 2022] GET /identity/v3/projects/a5677da7953148e5ad8ffb1bff5896ee => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:39.872350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65671dfb-37ca-4f49-9a23-207f2012669a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:39.970640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65671dfb-37ca-4f49-9a23-207f2012669a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:40.003184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e8424-375a-45f3-bca1-3cfa1e5ece5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:40.004063 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e8424-375a-45f3-bca1-3cfa1e5ece5f None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:40.004350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e8424-375a-45f3-bca1-3cfa1e5ece5f None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:40.004628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e8424-375a-45f3-bca1-3cfa1e5ece5f None admin] PATH_INFO: `/v3/projects/a5677da7953148e5ad8ffb1bff5896ee/users/3f2d9d25e4484ebd88a24dcdbeca02a1/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:40.006554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e8424-375a-45f3-bca1-3cfa1e5ece5f None admin] RBAC: Authorizing `identity:create_grant(project_id=a5677da7953148e5ad8ffb1bff5896ee, user_id=3f2d9d25e4484ebd88a24dcdbeca02a1, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:40.008011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e8424-375a-45f3-bca1-3cfa1e5ece5f None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:40.085229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 575/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:36:39 2022] PUT /identity/v3/projects/a5677da7953148e5ad8ffb1bff5896ee/users/3f2d9d25e4484ebd88a24dcdbeca02a1/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:36:42.079592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9b293-3bca-40a8-93ff-37ca3b1805dd None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:42.080038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9b293-3bca-40a8-93ff-37ca3b1805dd None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:42.080155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9b293-3bca-40a8-93ff-37ca3b1805dd None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:42.081545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 579/1154] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:42.086961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65b0f0-6d74-47bf-a3fd-d6535db9906a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:42.087384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65b0f0-6d74-47bf-a3fd-d6535db9906a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:42.087521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65b0f0-6d74-47bf-a3fd-d6535db9906a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:42.104610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-7f65b0f0-6d74-47bf-a3fd-d6535db9906a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:42.105795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7f65b0f0-6d74-47bf-a3fd-d6535db9906a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:42.116317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:42.172277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 576/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:42.195995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9b293-3bca-40a8-93ff-37ca3b1805dd None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:42.215504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:42.295846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9b293-3bca-40a8-93ff-37ca3b1805dd None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:42.328225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff71cf-45ff-43f7-985a-6af2611c6b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:42.329011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff71cf-45ff-43f7-985a-6af2611c6b2f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:42.329296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff71cf-45ff-43f7-985a-6af2611c6b2f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:42.329574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff71cf-45ff-43f7-985a-6af2611c6b2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:42.340405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff71cf-45ff-43f7-985a-6af2611c6b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-d7ff71cf-45ff-43f7-985a-6af2611c6b2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:42.347966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:42.349088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:42.350282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 580/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:36:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:42.355707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65b0f0-6d74-47bf-a3fd-d6535db9906a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:42.451875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65b0f0-6d74-47bf-a3fd-d6535db9906a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:42.485343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e2886-27ec-4e0d-b1ad-deae345d024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:42.486037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e2886-27ec-4e0d-b1ad-deae345d024a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:42.486329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e2886-27ec-4e0d-b1ad-deae345d024a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:42.486603 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e2886-27ec-4e0d-b1ad-deae345d024a None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:42.487346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e2886-27ec-4e0d-b1ad-deae345d024a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:42.487654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e2886-27ec-4e0d-b1ad-deae345d024a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:42.489029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e2886-27ec-4e0d-b1ad-deae345d024a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:42.496930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 577/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:36:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:42.502564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff71cf-45ff-43f7-985a-6af2611c6b2f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:42.579407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:42.600249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff71cf-45ff-43f7-985a-6af2611c6b2f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:42.633461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68734be6-3fc1-4548-8801-af361430b4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:42.634364 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68734be6-3fc1-4548-8801-af361430b4ce None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:42.634651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68734be6-3fc1-4548-8801-af361430b4ce None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:42.634930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68734be6-3fc1-4548-8801-af361430b4ce None admin] PATH_INFO: `/v3/users/3f2d9d25e4484ebd88a24dcdbeca02a1` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:42.636588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68734be6-3fc1-4548-8801-af361430b4ce None admin] RBAC: Authorizing `identity:get_user(user_id=3f2d9d25e4484ebd88a24dcdbeca02a1)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:42.638036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68734be6-3fc1-4548-8801-af361430b4ce None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:42.639653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 581/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:42 2022] GET /identity/v3/users/3f2d9d25e4484ebd88a24dcdbeca02a1 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:42.645323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e2886-27ec-4e0d-b1ad-deae345d024a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:42.656878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:42.744361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e2886-27ec-4e0d-b1ad-deae345d024a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:42.777555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90ac12-3ae3-4d86-b56b-2a953d7d0a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:42.778229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90ac12-3ae3-4d86-b56b-2a953d7d0a1e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:42.778518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90ac12-3ae3-4d86-b56b-2a953d7d0a1e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:42.778798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90ac12-3ae3-4d86-b56b-2a953d7d0a1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:42.845049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90ac12-3ae3-4d86-b56b-2a953d7d0a1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:42.846541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90ac12-3ae3-4d86-b56b-2a953d7d0a1e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:42.856225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:42.941763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 578/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:42.947594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68734be6-3fc1-4548-8801-af361430b4ce None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:43.045762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68734be6-3fc1-4548-8801-af361430b4ce None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:43.079452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbf364-13bf-4100-93c6-9abc42ffcd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:43.080305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbf364-13bf-4100-93c6-9abc42ffcd2f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:43.080835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbf364-13bf-4100-93c6-9abc42ffcd2f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:43.081033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbf364-13bf-4100-93c6-9abc42ffcd2f None admin] PATH_INFO: `/v3/projects/a5677da7953148e5ad8ffb1bff5896ee` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:43.082235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbf364-13bf-4100-93c6-9abc42ffcd2f None admin] RBAC: Authorizing `identity:get_project(project_id=a5677da7953148e5ad8ffb1bff5896ee)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:43.083653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbf364-13bf-4100-93c6-9abc42ffcd2f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:43.085339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 582/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:42 2022] GET /identity/v3/projects/a5677da7953148e5ad8ffb1bff5896ee => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:43.092061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90ac12-3ae3-4d86-b56b-2a953d7d0a1e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:43.187727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90ac12-3ae3-4d86-b56b-2a953d7d0a1e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:43.196844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:43.220566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b67ed-224c-4802-8aa3-b2b609d8bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:43.221273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b67ed-224c-4802-8aa3-b2b609d8bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:43.221553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b67ed-224c-4802-8aa3-b2b609d8bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:43.221852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b67ed-224c-4802-8aa3-b2b609d8bfb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:43.222665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b67ed-224c-4802-8aa3-b2b609d8bfb4 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=a5677da7953148e5ad8ffb1bff5896ee, user.id=3f2d9d25e4484ebd88a24dcdbeca02a1) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:43.222974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b67ed-224c-4802-8aa3-b2b609d8bfb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:43.224432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b67ed-224c-4802-8aa3-b2b609d8bfb4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:43.230943 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 579/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:36:43 2022] GET /identity/v3/role_assignments?user.id=3f2d9d25e4484ebd88a24dcdbeca02a1&scope.project.id=a5677da7953148e5ad8ffb1bff5896ee&role.id=abfb60dccca7470295258595a66a5d17 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:45.255051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6e121-9cbc-49c8-81f8-ee0ba040f713 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:45.255565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6e121-9cbc-49c8-81f8-ee0ba040f713 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:45.255680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6e121-9cbc-49c8-81f8-ee0ba040f713 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.256974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 583/1162] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:45.262891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9debf42-ccd4-405f-bb98-803f4a23fd05 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:45.263269 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9debf42-ccd4-405f-bb98-803f4a23fd05 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:45.263448 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9debf42-ccd4-405f-bb98-803f4a23fd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.279780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-b9debf42-ccd4-405f-bb98-803f4a23fd05 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:45.280952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b9debf42-ccd4-405f-bb98-803f4a23fd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:45.345569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 580/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:45.367574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6e121-9cbc-49c8-81f8-ee0ba040f713 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:45.382857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:45.467348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6e121-9cbc-49c8-81f8-ee0ba040f713 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:45.499910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12ea5-a750-4814-a1b6-5ab213fff233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:45.500554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12ea5-a750-4814-a1b6-5ab213fff233 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:45.500838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12ea5-a750-4814-a1b6-5ab213fff233 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:45.501117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf12ea5-a750-4814-a1b6-5ab213fff233 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.564704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf12ea5-a750-4814-a1b6-5ab213fff233 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:45.566423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf12ea5-a750-4814-a1b6-5ab213fff233 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:45.581383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:45.662359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 584/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:45.668306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9debf42-ccd4-405f-bb98-803f4a23fd05 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:45.766666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9debf42-ccd4-405f-bb98-803f4a23fd05 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:45.768164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:45.800006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c65f0f-2737-40ad-a196-8e4d419079f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:45.800936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65f0f-2737-40ad-a196-8e4d419079f8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:45.801243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65f0f-2737-40ad-a196-8e4d419079f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:45.801600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c65f0f-2737-40ad-a196-8e4d419079f8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.809386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c65f0f-2737-40ad-a196-8e4d419079f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:45.810863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c65f0f-2737-40ad-a196-8e4d419079f8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-b6c65f0f-2737-40ad-a196-8e4d419079f8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:45.825295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:45.826533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:36:45.827732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 581/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:36:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:45.832447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12ea5-a750-4814-a1b6-5ab213fff233 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:45.928325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf12ea5-a750-4814-a1b6-5ab213fff233 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:45.941906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:45.961224 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654f0c3-35ad-4fc5-b426-e846bc2633d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:45.961905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654f0c3-35ad-4fc5-b426-e846bc2633d0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:45.962194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654f0c3-35ad-4fc5-b426-e846bc2633d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:45.962466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654f0c3-35ad-4fc5-b426-e846bc2633d0 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:45.963190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654f0c3-35ad-4fc5-b426-e846bc2633d0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:45.964008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654f0c3-35ad-4fc5-b426-e846bc2633d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:45.965357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654f0c3-35ad-4fc5-b426-e846bc2633d0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:45.972672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 585/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 14 15:36:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:47.948236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9325b-63d7-4322-adfe-1878ae59053d None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:47.948795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9325b-63d7-4322-adfe-1878ae59053d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:47.948795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9325b-63d7-4322-adfe-1878ae59053d None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:47.950197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 582/1167] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:47.955245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd22192-70ff-4a72-877d-8894963cd1c5 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:47.955666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd22192-70ff-4a72-877d-8894963cd1c5 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:47.955822 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd22192-70ff-4a72-877d-8894963cd1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:47.972345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-afd22192-70ff-4a72-877d-8894963cd1c5 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:47.973530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-afd22192-70ff-4a72-877d-8894963cd1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:48.038666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 586/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:48.061972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9325b-63d7-4322-adfe-1878ae59053d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:48.162512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9325b-63d7-4322-adfe-1878ae59053d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:48.195378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03bb54-ed90-4e7c-85a1-bc20719a0e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:48.196029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03bb54-ed90-4e7c-85a1-bc20719a0e1b None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:48.196316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03bb54-ed90-4e7c-85a1-bc20719a0e1b None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:48.196596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03bb54-ed90-4e7c-85a1-bc20719a0e1b None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:48.197345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03bb54-ed90-4e7c-85a1-bc20719a0e1b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:48.198815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03bb54-ed90-4e7c-85a1-bc20719a0e1b None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:48.294608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 583/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Thu Jul 14 15:36:48 2022] POST /identity/v3/domains => generated 267 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:36:50.314166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216014f-259a-46cf-ad95-fccd8cd8db42 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:50.314921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216014f-259a-46cf-ad95-fccd8cd8db42 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:50.314921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216014f-259a-46cf-ad95-fccd8cd8db42 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:50.316307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 587/1170] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:50.321342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e649c-542e-43b5-979c-d4c44bf8d9df None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:50.321714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e649c-542e-43b5-979c-d4c44bf8d9df None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:50.321931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e649c-542e-43b5-979c-d4c44bf8d9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:50.338343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-1c6e649c-542e-43b5-979c-d4c44bf8d9df None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:50.339549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1c6e649c-542e-43b5-979c-d4c44bf8d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:50.356093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:50.405783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 584/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:50.429488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216014f-259a-46cf-ad95-fccd8cd8db42 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:50.529154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216014f-259a-46cf-ad95-fccd8cd8db42 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:50.538393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:50.562758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0a21d-6555-4e40-b32c-526ea2c021ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:50.563613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0a21d-6555-4e40-b32c-526ea2c021ef None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:50.563899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0a21d-6555-4e40-b32c-526ea2c021ef None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:50.564226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0a21d-6555-4e40-b32c-526ea2c021ef None admin] PATH_INFO: `/v3/domains/f8b91b0fefdd4e63bc5e1d2ae3db11e7` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:50.575081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0a21d-6555-4e40-b32c-526ea2c021ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=f8b91b0fefdd4e63bc5e1d2ae3db11e7)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:50.576518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0a21d-6555-4e40-b32c-526ea2c021ef None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:50.578188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 588/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Thu Jul 14 15:36:50 2022] GET /identity/v3/domains/f8b91b0fefdd4e63bc5e1d2ae3db11e7 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:50.583963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e649c-542e-43b5-979c-d4c44bf8d9df None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:50.681218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e649c-542e-43b5-979c-d4c44bf8d9df None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:50.714231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbda9b-c976-42d8-98df-150838e3f798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:50.714891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbda9b-c976-42d8-98df-150838e3f798 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:50.715178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbda9b-c976-42d8-98df-150838e3f798 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:50.715458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbda9b-c976-42d8-98df-150838e3f798 None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:50.716180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbda9b-c976-42d8-98df-150838e3f798 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:50.717615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbda9b-c976-42d8-98df-150838e3f798 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:50.805489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 585/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:36:50 2022] POST /identity/v3/projects => generated 361 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:36:52.842619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533aaa5-2aa9-4f26-b4a4-50a8be6ad59f None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:52.843551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533aaa5-2aa9-4f26-b4a4-50a8be6ad59f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:52.843551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533aaa5-2aa9-4f26-b4a4-50a8be6ad59f None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:52.844901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 589/1174] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:52.849984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61d78f-5915-4f5c-9cb6-76cf05f6b735 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:52.850440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61d78f-5915-4f5c-9cb6-76cf05f6b735 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:52.850542 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61d78f-5915-4f5c-9cb6-76cf05f6b735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:52.857067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:52.867878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-4a61d78f-5915-4f5c-9cb6-76cf05f6b735 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:52.869077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4a61d78f-5915-4f5c-9cb6-76cf05f6b735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:52.935598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 586/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:52.959668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533aaa5-2aa9-4f26-b4a4-50a8be6ad59f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:53.060312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533aaa5-2aa9-4f26-b4a4-50a8be6ad59f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:53.093869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996de1-5ebc-4230-8ec9-46430ff0c6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:53.094762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996de1-5ebc-4230-8ec9-46430ff0c6fc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:53.095071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996de1-5ebc-4230-8ec9-46430ff0c6fc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:53.095345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996de1-5ebc-4230-8ec9-46430ff0c6fc None admin] PATH_INFO: `/v3/domains/f8b91b0fefdd4e63bc5e1d2ae3db11e7` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:53.096518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996de1-5ebc-4230-8ec9-46430ff0c6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=f8b91b0fefdd4e63bc5e1d2ae3db11e7)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:53.097963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996de1-5ebc-4230-8ec9-46430ff0c6fc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:53.099576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 590/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Thu Jul 14 15:36:52 2022] GET /identity/v3/domains/f8b91b0fefdd4e63bc5e1d2ae3db11e7 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:53.105927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61d78f-5915-4f5c-9cb6-76cf05f6b735 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:53.201302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61d78f-5915-4f5c-9cb6-76cf05f6b735 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:53.233984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2d2e3-8b74-424f-bb2f-bee9975f46e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:53.234638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2d2e3-8b74-424f-bb2f-bee9975f46e4 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:53.234920 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2d2e3-8b74-424f-bb2f-bee9975f46e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:53.235197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2d2e3-8b74-424f-bb2f-bee9975f46e4 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:53.235897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2d2e3-8b74-424f-bb2f-bee9975f46e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:53.237422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2d2e3-8b74-424f-bb2f-bee9975f46e4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:53.243847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:53.350672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 587/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:36:53 2022] POST /identity/v3/users => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:36:55.407922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95dc98-94d4-4e45-9a74-1f046e3f4696 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.408493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95dc98-94d4-4e45-9a74-1f046e3f4696 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.408493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95dc98-94d4-4e45-9a74-1f046e3f4696 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.409768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 591/1178] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:55.415340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e570fa1-a78b-4f87-9a21-ff3daf25fedf None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.415771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e570fa1-a78b-4f87-9a21-ff3daf25fedf None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.415903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e570fa1-a78b-4f87-9a21-ff3daf25fedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.436223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5e570fa1-a78b-4f87-9a21-ff3daf25fedf None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:55.437408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5e570fa1-a78b-4f87-9a21-ff3daf25fedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:55.511377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 588/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:55.535090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95dc98-94d4-4e45-9a74-1f046e3f4696 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:55.572179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:55.641703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95dc98-94d4-4e45-9a74-1f046e3f4696 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:55.678591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82867fa9-9746-4c77-864d-446e7b9e1fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:55.679320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867fa9-9746-4c77-864d-446e7b9e1fad None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.679605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867fa9-9746-4c77-864d-446e7b9e1fad None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.679883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867fa9-9746-4c77-864d-446e7b9e1fad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.691313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82867fa9-9746-4c77-864d-446e7b9e1fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-82867fa9-9746-4c77-864d-446e7b9e1fad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:55.699836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:55.701026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:55.702182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:55.702924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 592/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:36:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:55.708193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e570fa1-a78b-4f87-9a21-ff3daf25fedf None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:55.805641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e570fa1-a78b-4f87-9a21-ff3daf25fedf None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:55.825576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:55.841236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59c87d-3e83-43f8-a95a-94384a9cf954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:55.841939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59c87d-3e83-43f8-a95a-94384a9cf954 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.842227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59c87d-3e83-43f8-a95a-94384a9cf954 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.842507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59c87d-3e83-43f8-a95a-94384a9cf954 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:55.843258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59c87d-3e83-43f8-a95a-94384a9cf954 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:55.843571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59c87d-3e83-43f8-a95a-94384a9cf954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:55.844949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59c87d-3e83-43f8-a95a-94384a9cf954 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:55.853315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 589/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:36:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:55.859771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82867fa9-9746-4c77-864d-446e7b9e1fad None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:55.952437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:55.958076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82867fa9-9746-4c77-864d-446e7b9e1fad None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:55.992086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc0a80-2a85-4d38-9165-1b8bd749ee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:55.993092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc0a80-2a85-4d38-9165-1b8bd749ee66 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:55.993373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc0a80-2a85-4d38-9165-1b8bd749ee66 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:55.993643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc0a80-2a85-4d38-9165-1b8bd749ee66 None admin] PATH_INFO: `/v3/users/ad6d27d4001849678e6ab9de0fa752ce` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.006658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc0a80-2a85-4d38-9165-1b8bd749ee66 None admin] RBAC: Authorizing `identity:get_user(user_id=ad6d27d4001849678e6ab9de0fa752ce)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:56.008109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc0a80-2a85-4d38-9165-1b8bd749ee66 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:56.009764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 593/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:55 2022] GET /identity/v3/users/ad6d27d4001849678e6ab9de0fa752ce => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:56.016136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59c87d-3e83-43f8-a95a-94384a9cf954 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:56.118916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59c87d-3e83-43f8-a95a-94384a9cf954 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:56.153494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e1c73-ed12-4781-9156-223b1c2d40f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:56.154175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e1c73-ed12-4781-9156-223b1c2d40f3 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.154458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e1c73-ed12-4781-9156-223b1c2d40f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.154730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e1c73-ed12-4781-9156-223b1c2d40f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.202626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:56.220976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e1c73-ed12-4781-9156-223b1c2d40f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:56.222454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e1c73-ed12-4781-9156-223b1c2d40f3 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:56.317623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 590/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:36:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:36:56.324047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc0a80-2a85-4d38-9165-1b8bd749ee66 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:56.420878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc0a80-2a85-4d38-9165-1b8bd749ee66 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:56.454342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf5c30-7a2b-4f4e-bd62-3399b47f42ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:56.455138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf5c30-7a2b-4f4e-bd62-3399b47f42ce None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.455421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf5c30-7a2b-4f4e-bd62-3399b47f42ce None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.455695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf5c30-7a2b-4f4e-bd62-3399b47f42ce None admin] PATH_INFO: `/v3/projects/05b742a832e0449a85b6f3ef590256ff` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.456861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf5c30-7a2b-4f4e-bd62-3399b47f42ce None admin] RBAC: Authorizing `identity:get_project(project_id=05b742a832e0449a85b6f3ef590256ff)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:56.458320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf5c30-7a2b-4f4e-bd62-3399b47f42ce None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:56.459991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 594/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:56 2022] GET /identity/v3/projects/05b742a832e0449a85b6f3ef590256ff => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:56.466169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e1c73-ed12-4781-9156-223b1c2d40f3 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:56.546445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:56.564299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e1c73-ed12-4781-9156-223b1c2d40f3 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:56.598143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230436b-154f-4b73-b36c-850c45fc3b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:56.598799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230436b-154f-4b73-b36c-850c45fc3b1a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:56.599086 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230436b-154f-4b73-b36c-850c45fc3b1a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:56.599370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230436b-154f-4b73-b36c-850c45fc3b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:56.600199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230436b-154f-4b73-b36c-850c45fc3b1a None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=05b742a832e0449a85b6f3ef590256ff, user.id=ad6d27d4001849678e6ab9de0fa752ce) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:56.600515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230436b-154f-4b73-b36c-850c45fc3b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:56.601982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230436b-154f-4b73-b36c-850c45fc3b1a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:56.607485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 591/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:36:56 2022] GET /identity/v3/role_assignments?user.id=ad6d27d4001849678e6ab9de0fa752ce&scope.project.id=05b742a832e0449a85b6f3ef590256ff&role.id=abfb60dccca7470295258595a66a5d17 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:58.571353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1606c57-0fbc-4842-b4b9-9819c16a25dc None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.571913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1606c57-0fbc-4842-b4b9-9819c16a25dc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.571913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1606c57-0fbc-4842-b4b9-9819c16a25dc None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.573791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 595/1186] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:36:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:36:58.578937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff103-b790-4020-aeba-6c0bdd207b34 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.579337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff103-b790-4020-aeba-6c0bdd207b34 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.579488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff103-b790-4020-aeba-6c0bdd207b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.596537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5a0ff103-b790-4020-aeba-6c0bdd207b34 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:36:58.597716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5a0ff103-b790-4020-aeba-6c0bdd207b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:36:58.663145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 592/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:36:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:36:58.685764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1606c57-0fbc-4842-b4b9-9819c16a25dc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:58.710922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:58.785311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1606c57-0fbc-4842-b4b9-9819c16a25dc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:58.817923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d6aa7-fbd2-4d7c-bf68-632ae09c1b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:58.818706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d6aa7-fbd2-4d7c-bf68-632ae09c1b7e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.818991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d6aa7-fbd2-4d7c-bf68-632ae09c1b7e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.819271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d6aa7-fbd2-4d7c-bf68-632ae09c1b7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.830174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d6aa7-fbd2-4d7c-bf68-632ae09c1b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-6d6d6aa7-fbd2-4d7c-bf68-632ae09c1b7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:36:58.837670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:36:58.838831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:36:58.839884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 596/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:36:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:36:58.845128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ff103-b790-4020-aeba-6c0bdd207b34 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:58.918969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:58.941895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ff103-b790-4020-aeba-6c0bdd207b34 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:58.975040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fdebf-7f25-4b1a-9193-2b55da5f6785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:58.975699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fdebf-7f25-4b1a-9193-2b55da5f6785 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:58.975984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fdebf-7f25-4b1a-9193-2b55da5f6785 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:58.976261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fdebf-7f25-4b1a-9193-2b55da5f6785 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:58.976999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fdebf-7f25-4b1a-9193-2b55da5f6785 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:36:58.977306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fdebf-7f25-4b1a-9193-2b55da5f6785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:58.978692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fdebf-7f25-4b1a-9193-2b55da5f6785 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:58.986850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 593/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:36:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:58.992532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d6aa7-fbd2-4d7c-bf68-632ae09c1b7e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:59.074966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:36:59.092968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d6aa7-fbd2-4d7c-bf68-632ae09c1b7e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:59.126236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fe3b7-c2ef-4ca0-8dbe-f3ca10548277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:59.127111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fe3b7-c2ef-4ca0-8dbe-f3ca10548277 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:59.127394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fe3b7-c2ef-4ca0-8dbe-f3ca10548277 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:59.127672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fe3b7-c2ef-4ca0-8dbe-f3ca10548277 None admin] PATH_INFO: `/v3/users/ad6d27d4001849678e6ab9de0fa752ce` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:59.128850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fe3b7-c2ef-4ca0-8dbe-f3ca10548277 None admin] RBAC: Authorizing `identity:get_user(user_id=ad6d27d4001849678e6ab9de0fa752ce)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:59.130308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fe3b7-c2ef-4ca0-8dbe-f3ca10548277 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:59.131934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 597/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:36:58 2022] GET /identity/v3/users/ad6d27d4001849678e6ab9de0fa752ce => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:59.138245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fdebf-7f25-4b1a-9193-2b55da5f6785 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:59.236633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fdebf-7f25-4b1a-9193-2b55da5f6785 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:59.269519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49569c9-746f-4b20-9895-6dbb8c76088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:59.270396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49569c9-746f-4b20-9895-6dbb8c76088a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:59.270682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49569c9-746f-4b20-9895-6dbb8c76088a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:59.270959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49569c9-746f-4b20-9895-6dbb8c76088a None admin] PATH_INFO: `/v3/projects/05b742a832e0449a85b6f3ef590256ff` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:59.272128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49569c9-746f-4b20-9895-6dbb8c76088a None admin] RBAC: Authorizing `identity:get_project(project_id=05b742a832e0449a85b6f3ef590256ff)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:59.273554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49569c9-746f-4b20-9895-6dbb8c76088a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:59.275247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 594/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:36:59 2022] GET /identity/v3/projects/05b742a832e0449a85b6f3ef590256ff => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:36:59.280914 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fe3b7-c2ef-4ca0-8dbe-f3ca10548277 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:36:59.377193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fe3b7-c2ef-4ca0-8dbe-f3ca10548277 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:36:59.409526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51e959-d2a9-4020-8f85-898648bd5fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:36:59.410422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51e959-d2a9-4020-8f85-898648bd5fd7 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:36:59.410703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51e959-d2a9-4020-8f85-898648bd5fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:36:59.410979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51e959-d2a9-4020-8f85-898648bd5fd7 None admin] PATH_INFO: `/v3/projects/05b742a832e0449a85b6f3ef590256ff/users/ad6d27d4001849678e6ab9de0fa752ce/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:36:59.412906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51e959-d2a9-4020-8f85-898648bd5fd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=05b742a832e0449a85b6f3ef590256ff, user_id=ad6d27d4001849678e6ab9de0fa752ce, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:36:59.414347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51e959-d2a9-4020-8f85-898648bd5fd7 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:36:59.495846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 598/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:36:59 2022] PUT /identity/v3/projects/05b742a832e0449a85b6f3ef590256ff/users/ad6d27d4001849678e6ab9de0fa752ce/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:37:01.479692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e4323-db1f-499f-b0b5-9bc06f7b2246 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:01.480203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e4323-db1f-499f-b0b5-9bc06f7b2246 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:01.480297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e4323-db1f-499f-b0b5-9bc06f7b2246 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:01.481649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 595/1193] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:37:01.486936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276f14b-8e45-41ad-b19c-9aff24102606 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:01.487357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276f14b-8e45-41ad-b19c-9aff24102606 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:01.487501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276f14b-8e45-41ad-b19c-9aff24102606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:01.505247 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-1276f14b-8e45-41ad-b19c-9aff24102606 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:37:01.506456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1276f14b-8e45-41ad-b19c-9aff24102606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:37:01.509134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:01.571046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 599/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:37:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:37:01.594799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e4323-db1f-499f-b0b5-9bc06f7b2246 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:01.604648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:01.696714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e4323-db1f-499f-b0b5-9bc06f7b2246 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:01.729887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b53a72-6ed7-42bf-ad6a-19bd3e947f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:01.730635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b53a72-6ed7-42bf-ad6a-19bd3e947f3d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:01.730921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b53a72-6ed7-42bf-ad6a-19bd3e947f3d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:01.731209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b53a72-6ed7-42bf-ad6a-19bd3e947f3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:01.741532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b53a72-6ed7-42bf-ad6a-19bd3e947f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-84b53a72-6ed7-42bf-ad6a-19bd3e947f3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:37:01.749639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:37:01.750799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:37:01.752036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 596/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:37:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:37:01.756774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276f14b-8e45-41ad-b19c-9aff24102606 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:01.852948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276f14b-8e45-41ad-b19c-9aff24102606 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:01.885498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848e9de-1341-440b-8de4-14cbb3de71cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:01.886188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848e9de-1341-440b-8de4-14cbb3de71cf None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:01.886470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848e9de-1341-440b-8de4-14cbb3de71cf None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:01.886743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848e9de-1341-440b-8de4-14cbb3de71cf None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:01.887482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848e9de-1341-440b-8de4-14cbb3de71cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:01.887786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848e9de-1341-440b-8de4-14cbb3de71cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:01.889131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848e9de-1341-440b-8de4-14cbb3de71cf None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:01.896923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 600/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:37:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:01.903055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b53a72-6ed7-42bf-ad6a-19bd3e947f3d None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:01.967859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:02.001995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b53a72-6ed7-42bf-ad6a-19bd3e947f3d None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:02.037203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c648-93ed-4ea5-86f6-79f21e76845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:02.038110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c648-93ed-4ea5-86f6-79f21e76845e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:02.038392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c648-93ed-4ea5-86f6-79f21e76845e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:02.038663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c648-93ed-4ea5-86f6-79f21e76845e None admin] PATH_INFO: `/v3/users/ad6d27d4001849678e6ab9de0fa752ce` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:02.039857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289c648-93ed-4ea5-86f6-79f21e76845e None admin] RBAC: Authorizing `identity:get_user(user_id=ad6d27d4001849678e6ab9de0fa752ce)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:02.042105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289c648-93ed-4ea5-86f6-79f21e76845e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:02.043434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 597/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 14 15:37:01 2022] GET /identity/v3/users/ad6d27d4001849678e6ab9de0fa752ce => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:02.049303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848e9de-1341-440b-8de4-14cbb3de71cf None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:02.052169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:02.147160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848e9de-1341-440b-8de4-14cbb3de71cf None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:02.179207 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad8dab-5720-48b7-8082-449319fe6f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:02.179849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad8dab-5720-48b7-8082-449319fe6f61 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:02.180126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad8dab-5720-48b7-8082-449319fe6f61 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:02.180396 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad8dab-5720-48b7-8082-449319fe6f61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:02.244382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad8dab-5720-48b7-8082-449319fe6f61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:02.246376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad8dab-5720-48b7-8082-449319fe6f61 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:02.248084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:02.340048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 601/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:37:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:37:02.345962 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c648-93ed-4ea5-86f6-79f21e76845e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:02.443101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c648-93ed-4ea5-86f6-79f21e76845e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:02.476118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb5633-b9ce-4bc2-ba55-2d370256d7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:02.476963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb5633-b9ce-4bc2-ba55-2d370256d7a1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:02.477245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb5633-b9ce-4bc2-ba55-2d370256d7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:02.477521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb5633-b9ce-4bc2-ba55-2d370256d7a1 None admin] PATH_INFO: `/v3/projects/05b742a832e0449a85b6f3ef590256ff` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:02.478719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb5633-b9ce-4bc2-ba55-2d370256d7a1 None admin] RBAC: Authorizing `identity:get_project(project_id=05b742a832e0449a85b6f3ef590256ff)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:02.480688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb5633-b9ce-4bc2-ba55-2d370256d7a1 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:02.482380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 598/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 14 15:37:02 2022] GET /identity/v3/projects/05b742a832e0449a85b6f3ef590256ff => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:02.488329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad8dab-5720-48b7-8082-449319fe6f61 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:37:02.584143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad8dab-5720-48b7-8082-449319fe6f61 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:37:02.585635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:02.617785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77f780-f9bc-4cc1-98be-25d0adeca85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:37:02.618400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77f780-f9bc-4cc1-98be-25d0adeca85a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:37:02.618684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77f780-f9bc-4cc1-98be-25d0adeca85a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:37:02.618965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77f780-f9bc-4cc1-98be-25d0adeca85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:37:02.620295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77f780-f9bc-4cc1-98be-25d0adeca85a None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=05b742a832e0449a85b6f3ef590256ff, user.id=ad6d27d4001849678e6ab9de0fa752ce) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:37:02.620604 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77f780-f9bc-4cc1-98be-25d0adeca85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:37:02.622022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77f780-f9bc-4cc1-98be-25d0adeca85a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:37:02.627667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 602/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:37:02 2022] GET /identity/v3/role_assignments?user.id=ad6d27d4001849678e6ab9de0fa752ce&scope.project.id=05b742a832e0449a85b6f3ef590256ff&role.id=abfb60dccca7470295258595a66a5d17 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:37:12.472044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:37:12.625084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=17 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:15.493236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd9252-9a97-4593-a24f-f41fc13efc97 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:15.493803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd9252-9a97-4593-a24f-f41fc13efc97 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:15.493803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd9252-9a97-4593-a24f-f41fc13efc97 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:15.495236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 599/1201] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:15.500784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1aa3-d28c-4918-8ce6-47ebd295429e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:15.502052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1aa3-d28c-4918-8ce6-47ebd295429e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:15.502052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1aa3-d28c-4918-8ce6-47ebd295429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:15.518394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b81a1aa3-d28c-4918-8ce6-47ebd295429e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:15.519589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b81a1aa3-d28c-4918-8ce6-47ebd295429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:15.586883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 603/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:15.611365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd9252-9a97-4593-a24f-f41fc13efc97 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:15.712841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd9252-9a97-4593-a24f-f41fc13efc97 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:15.747601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b86e1-78d6-4d38-89a0-83eb6ccdf4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:15.748337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b86e1-78d6-4d38-89a0-83eb6ccdf4ed None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:15.748619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b86e1-78d6-4d38-89a0-83eb6ccdf4ed None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:15.748895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b86e1-78d6-4d38-89a0-83eb6ccdf4ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:15.758926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b86e1-78d6-4d38-89a0-83eb6ccdf4ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:15.760371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b86e1-78d6-4d38-89a0-83eb6ccdf4ed None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-5d9b86e1-78d6-4d38-89a0-83eb6ccdf4ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:15.778057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:15.779195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:15.780417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:15.780417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:15.780417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:15.780417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:15.780417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 600/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:15.785463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1aa3-d28c-4918-8ce6-47ebd295429e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:15.884300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1aa3-d28c-4918-8ce6-47ebd295429e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:15.912762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:15.918485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153eec62-3a50-487f-8b28-701ea48dc585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:15.919155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153eec62-3a50-487f-8b28-701ea48dc585 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:15.919440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153eec62-3a50-487f-8b28-701ea48dc585 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:15.919733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153eec62-3a50-487f-8b28-701ea48dc585 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:15.920437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153eec62-3a50-487f-8b28-701ea48dc585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:15.920745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153eec62-3a50-487f-8b28-701ea48dc585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:15.922144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153eec62-3a50-487f-8b28-701ea48dc585 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:15.932991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 604/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:15.938698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b86e1-78d6-4d38-89a0-83eb6ccdf4ed None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:15.992621 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:16.039352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b86e1-78d6-4d38-89a0-83eb6ccdf4ed None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:16.071158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac780e-4296-47c7-bed3-3a8bc682cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:16.071827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac780e-4296-47c7-bed3-3a8bc682cbb0 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:16.072107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac780e-4296-47c7-bed3-3a8bc682cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:16.072424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac780e-4296-47c7-bed3-3a8bc682cbb0 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:16.073097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac780e-4296-47c7-bed3-3a8bc682cbb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:16.075063 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac780e-4296-47c7-bed3-3a8bc682cbb0 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:16.174251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 601/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 14 15:38:15 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:18.225933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba029d-8cbc-4d0f-8f15-d3daac17c2e4 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:18.226474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba029d-8cbc-4d0f-8f15-d3daac17c2e4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:18.226474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba029d-8cbc-4d0f-8f15-d3daac17c2e4 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:18.227922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 605/1206] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:18.233477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b11f1-fe78-43bc-8a3e-63047da2d10f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:18.233944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b11f1-fe78-43bc-8a3e-63047da2d10f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:18.234076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b11f1-fe78-43bc-8a3e-63047da2d10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:18.251206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-976b11f1-fe78-43bc-8a3e-63047da2d10f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:18.252402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-976b11f1-fe78-43bc-8a3e-63047da2d10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:18.318850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 602/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:18.342249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba029d-8cbc-4d0f-8f15-d3daac17c2e4 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:18.442790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba029d-8cbc-4d0f-8f15-d3daac17c2e4 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:18.474800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3196b-6752-4118-a80a-4778325bc13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:18.475592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3196b-6752-4118-a80a-4778325bc13e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:18.475876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3196b-6752-4118-a80a-4778325bc13e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:18.476154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3196b-6752-4118-a80a-4778325bc13e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:18.488109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3196b-6752-4118-a80a-4778325bc13e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-1cd3196b-6752-4118-a80a-4778325bc13e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:38:18.495777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:18.497601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:18.501701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 606/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:38:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:18.503766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b11f1-fe78-43bc-8a3e-63047da2d10f None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:18.542565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:18.600423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b11f1-fe78-43bc-8a3e-63047da2d10f None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:18.633468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a20eb-7005-48fd-b002-e448bfce4946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:18.634157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a20eb-7005-48fd-b002-e448bfce4946 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:18.634438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a20eb-7005-48fd-b002-e448bfce4946 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:18.634718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a20eb-7005-48fd-b002-e448bfce4946 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:18.635453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a20eb-7005-48fd-b002-e448bfce4946 None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:18.635756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a20eb-7005-48fd-b002-e448bfce4946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:18.637152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a20eb-7005-48fd-b002-e448bfce4946 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:18.644894 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 603/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:38:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:18.650810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3196b-6752-4118-a80a-4778325bc13e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:18.701154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:18.749940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3196b-6752-4118-a80a-4778325bc13e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:18.782822 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ac680-cbd5-4802-bc91-723c53b57e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:18.783609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ac680-cbd5-4802-bc91-723c53b57e0e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:18.783896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ac680-cbd5-4802-bc91-723c53b57e0e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:18.784175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ac680-cbd5-4802-bc91-723c53b57e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:18.795265 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ac680-cbd5-4802-bc91-723c53b57e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:18.796705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ac680-cbd5-4802-bc91-723c53b57e0e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-351ac680-cbd5-4802-bc91-723c53b57e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:18.814199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:18.815143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:18.816091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:18.816091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:18.816091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:18.816091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:18.816091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 607/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:38:18.821146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a20eb-7005-48fd-b002-e448bfce4946 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:18.917596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:18.920915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a20eb-7005-48fd-b002-e448bfce4946 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:18.954445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6640a7-66bd-4ade-acd2-ea23f1bc98d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:18.955327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6640a7-66bd-4ade-acd2-ea23f1bc98d5 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:18.955615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6640a7-66bd-4ade-acd2-ea23f1bc98d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:18.955889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6640a7-66bd-4ade-acd2-ea23f1bc98d5 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:18.960417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6640a7-66bd-4ade-acd2-ea23f1bc98d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:18.960759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6640a7-66bd-4ade-acd2-ea23f1bc98d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:18.962244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6640a7-66bd-4ade-acd2-ea23f1bc98d5 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:18.974418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 604/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:18.979883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ac680-cbd5-4802-bc91-723c53b57e0e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.081718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ac680-cbd5-4802-bc91-723c53b57e0e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.083265 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:19.114787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1106a1-6781-4ea4-a9e6-c18ff8336f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.115589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1106a1-6781-4ea4-a9e6-c18ff8336f0c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.115870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1106a1-6781-4ea4-a9e6-c18ff8336f0c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.116145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1106a1-6781-4ea4-a9e6-c18ff8336f0c None admin] PATH_INFO: `/v3/users/placement` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.123893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1106a1-6781-4ea4-a9e6-c18ff8336f0c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.125319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1106a1-6781-4ea4-a9e6-c18ff8336f0c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-8c1106a1-6781-4ea4-a9e6-c18ff8336f0c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:38:19.139326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:19.140398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:19.141724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 608/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:19.146294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6640a7-66bd-4ade-acd2-ea23f1bc98d5 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.246688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6640a7-66bd-4ade-acd2-ea23f1bc98d5 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.280262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d6b67-0025-4807-86bb-a8d8b7eecd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.280936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d6b67-0025-4807-86bb-a8d8b7eecd75 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.281216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d6b67-0025-4807-86bb-a8d8b7eecd75 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.281487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d6b67-0025-4807-86bb-a8d8b7eecd75 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.282673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d6b67-0025-4807-86bb-a8d8b7eecd75 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:19.282673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d6b67-0025-4807-86bb-a8d8b7eecd75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.284077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d6b67-0025-4807-86bb-a8d8b7eecd75 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.301717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 605/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 14 15:38:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:19.307560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1106a1-6781-4ea4-a9e6-c18ff8336f0c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.407565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1106a1-6781-4ea4-a9e6-c18ff8336f0c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.440384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8eab7-aea9-4834-b09a-b01d52371c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.441029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8eab7-aea9-4834-b09a-b01d52371c4e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.441312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8eab7-aea9-4834-b09a-b01d52371c4e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.441585 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8eab7-aea9-4834-b09a-b01d52371c4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.470700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:19.506921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8eab7-aea9-4834-b09a-b01d52371c4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.508297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8eab7-aea9-4834-b09a-b01d52371c4e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.603330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 609/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:38:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:38:19.609767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d6b67-0025-4807-86bb-a8d8b7eecd75 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.628259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:19.708251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d6b67-0025-4807-86bb-a8d8b7eecd75 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.742304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6ef5-d3ec-4801-884a-7663467b897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.743100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6ef5-d3ec-4801-884a-7663467b897a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.743406 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6ef5-d3ec-4801-884a-7663467b897a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.743681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca6ef5-d3ec-4801-884a-7663467b897a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.753786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca6ef5-d3ec-4801-884a-7663467b897a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.755267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca6ef5-d3ec-4801-884a-7663467b897a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-a4ca6ef5-d3ec-4801-884a-7663467b897a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:19.773061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:19.774152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:19.775106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:19.775106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:19.775106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:19.775106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:19.775106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 606/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:19.780550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8eab7-aea9-4834-b09a-b01d52371c4e None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.840879 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:19.877179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8eab7-aea9-4834-b09a-b01d52371c4e None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:19.910061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839a818-cdac-43a4-94a3-c8cfaf5278db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:19.910717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839a818-cdac-43a4-94a3-c8cfaf5278db None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:19.910999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839a818-cdac-43a4-94a3-c8cfaf5278db None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:19.911274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839a818-cdac-43a4-94a3-c8cfaf5278db None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:19.911988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839a818-cdac-43a4-94a3-c8cfaf5278db None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:19.912291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839a818-cdac-43a4-94a3-c8cfaf5278db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:19.913718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839a818-cdac-43a4-94a3-c8cfaf5278db None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:19.924570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 610/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:19.930384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6ef5-d3ec-4801-884a-7663467b897a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:19.999909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:20.029899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca6ef5-d3ec-4801-884a-7663467b897a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.062961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142c45-b45c-40bd-82c5-84ac319a5221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.063747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142c45-b45c-40bd-82c5-84ac319a5221 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.064280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142c45-b45c-40bd-82c5-84ac319a5221 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.064552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142c45-b45c-40bd-82c5-84ac319a5221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.071051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a142c45-b45c-40bd-82c5-84ac319a5221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.072497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a142c45-b45c-40bd-82c5-84ac319a5221 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-1a142c45-b45c-40bd-82c5-84ac319a5221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:38:20.086530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:20.087514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:20.088556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:38:20.088556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:38:20.088556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:38:20.088556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:38:20.088556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:38:20.088556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:38:20.088556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:20.088556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:20.088556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 607/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:20.093765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839a818-cdac-43a4-94a3-c8cfaf5278db None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:20.193348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839a818-cdac-43a4-94a3-c8cfaf5278db None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.215978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:20.227013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e383ee3-3ea3-404a-8062-b5d41e2e5b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.227672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e383ee3-3ea3-404a-8062-b5d41e2e5b3b None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.227957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e383ee3-3ea3-404a-8062-b5d41e2e5b3b None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.228233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e383ee3-3ea3-404a-8062-b5d41e2e5b3b None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.229017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e383ee3-3ea3-404a-8062-b5d41e2e5b3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:20.229322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e383ee3-3ea3-404a-8062-b5d41e2e5b3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.230805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e383ee3-3ea3-404a-8062-b5d41e2e5b3b None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.242361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 611/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:38:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:20.249041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142c45-b45c-40bd-82c5-84ac319a5221 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:20.349601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142c45-b45c-40bd-82c5-84ac319a5221 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:20.377524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:20.383231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd6e40-d6ea-4208-be53-3d88adac5a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:20.383885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd6e40-d6ea-4208-be53-3d88adac5a36 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:20.384172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd6e40-d6ea-4208-be53-3d88adac5a36 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:20.384452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd6e40-d6ea-4208-be53-3d88adac5a36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:20.385284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd6e40-d6ea-4208-be53-3d88adac5a36 None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=97524081e59f4775b83d82edf737810e) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:20.385598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd6e40-d6ea-4208-be53-3d88adac5a36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:20.387584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd6e40-d6ea-4208-be53-3d88adac5a36 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:20.393117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 608/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:38:20 2022] GET /identity/v3/role_assignments?user.id=97524081e59f4775b83d82edf737810e&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:22.378269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276becd-61bd-453c-91d7-595a4cce09c3 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.378781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276becd-61bd-453c-91d7-595a4cce09c3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.378781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276becd-61bd-453c-91d7-595a4cce09c3 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.380135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 612/1220] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:22.385465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad570b21-68ef-4075-8c19-bf92ddabd194 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.387347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad570b21-68ef-4075-8c19-bf92ddabd194 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.387347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad570b21-68ef-4075-8c19-bf92ddabd194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.403396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ad570b21-68ef-4075-8c19-bf92ddabd194 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:22.404583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ad570b21-68ef-4075-8c19-bf92ddabd194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:22.472033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 609/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:22.494306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276becd-61bd-453c-91d7-595a4cce09c3 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:22.594628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276becd-61bd-453c-91d7-595a4cce09c3 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:22.627803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64e52e-43fa-4ded-ad89-46071e4704fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:22.628633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64e52e-43fa-4ded-ad89-46071e4704fc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.628867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64e52e-43fa-4ded-ad89-46071e4704fc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.629142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64e52e-43fa-4ded-ad89-46071e4704fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.639549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64e52e-43fa-4ded-ad89-46071e4704fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-8e64e52e-43fa-4ded-ad89-46071e4704fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:38:22.647505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:22.664985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:22.666088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 613/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:38:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:22.666945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad570b21-68ef-4075-8c19-bf92ddabd194 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:22.755255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad570b21-68ef-4075-8c19-bf92ddabd194 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:22.778820 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:22.789083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd5e29-087d-44e4-9000-76650d1e6f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:22.789756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd5e29-087d-44e4-9000-76650d1e6f21 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.790081 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd5e29-087d-44e4-9000-76650d1e6f21 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.790392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd5e29-087d-44e4-9000-76650d1e6f21 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.791104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd5e29-087d-44e4-9000-76650d1e6f21 None admin] RBAC: Adding query filter params (name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:22.791418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd5e29-087d-44e4-9000-76650d1e6f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:22.793282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd5e29-087d-44e4-9000-76650d1e6f21 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:22.801129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 610/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:22.806939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64e52e-43fa-4ded-ad89-46071e4704fc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:22.851118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:22.906181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64e52e-43fa-4ded-ad89-46071e4704fc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:22.939912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac6ff6-0584-401a-9bdb-f69595c7ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:22.940695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac6ff6-0584-401a-9bdb-f69595c7ea4d None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:22.940974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac6ff6-0584-401a-9bdb-f69595c7ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:22.941263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac6ff6-0584-401a-9bdb-f69595c7ea4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:22.951513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac6ff6-0584-401a-9bdb-f69595c7ea4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:22.952936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac6ff6-0584-401a-9bdb-f69595c7ea4d None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-81ac6ff6-0584-401a-9bdb-f69595c7ea4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:22.969621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:22.970809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:22.971841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:22.971841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:22.971841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:22.971841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:22.971841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 614/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:22.976787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd5e29-087d-44e4-9000-76650d1e6f21 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.077608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd5e29-087d-44e4-9000-76650d1e6f21 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.111181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19d4de-c07e-4e1b-a14e-62da808668d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.111838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d4de-c07e-4e1b-a14e-62da808668d1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.112123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d4de-c07e-4e1b-a14e-62da808668d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.112412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d4de-c07e-4e1b-a14e-62da808668d1 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.113132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19d4de-c07e-4e1b-a14e-62da808668d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:23.113443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19d4de-c07e-4e1b-a14e-62da808668d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.114840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19d4de-c07e-4e1b-a14e-62da808668d1 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.126850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 611/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:23.132662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac6ff6-0584-401a-9bdb-f69595c7ea4d None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.226963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:23.235562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac6ff6-0584-401a-9bdb-f69595c7ea4d None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.267698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf98777-9d30-42df-8f69-9b8a37966892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.268486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf98777-9d30-42df-8f69-9b8a37966892 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.268772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf98777-9d30-42df-8f69-9b8a37966892 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.269050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf98777-9d30-42df-8f69-9b8a37966892 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.276734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf98777-9d30-42df-8f69-9b8a37966892 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.278748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf98777-9d30-42df-8f69-9b8a37966892 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-ebf98777-9d30-42df-8f69-9b8a37966892 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:38:23.292806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:23.294435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:23.295594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 615/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:23.300645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19d4de-c07e-4e1b-a14e-62da808668d1 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.347305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:23.398605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19d4de-c07e-4e1b-a14e-62da808668d1 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.431475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf5dc-fb05-4a85-8bee-0cda63506225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.432167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf5dc-fb05-4a85-8bee-0cda63506225 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.432456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf5dc-fb05-4a85-8bee-0cda63506225 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.432735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf5dc-fb05-4a85-8bee-0cda63506225 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.433593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bf5dc-fb05-4a85-8bee-0cda63506225 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:23.433936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bf5dc-fb05-4a85-8bee-0cda63506225 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.435365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bf5dc-fb05-4a85-8bee-0cda63506225 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.453310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 612/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:38:23.459206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf98777-9d30-42df-8f69-9b8a37966892 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.557807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf98777-9d30-42df-8f69-9b8a37966892 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.591665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141d550-024a-4b6b-91e6-a82e520be6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.592447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141d550-024a-4b6b-91e6-a82e520be6d0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.592732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141d550-024a-4b6b-91e6-a82e520be6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.593006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8141d550-024a-4b6b-91e6-a82e520be6d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.603442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141d550-024a-4b6b-91e6-a82e520be6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.604876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8141d550-024a-4b6b-91e6-a82e520be6d0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-8141d550-024a-4b6b-91e6-a82e520be6d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:23.622052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:23.623113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:23.624217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:23.624217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:23.624217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:23.624217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:23.624217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 616/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:23.628956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf5dc-fb05-4a85-8bee-0cda63506225 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.728108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf5dc-fb05-4a85-8bee-0cda63506225 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.730404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:23.761513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2b296-c832-429b-b630-42858745f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.762343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2b296-c832-429b-b630-42858745f485 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.762628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2b296-c832-429b-b630-42858745f485 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.762903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2b296-c832-429b-b630-42858745f485 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.763632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2b296-c832-429b-b630-42858745f485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:23.763939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2b296-c832-429b-b630-42858745f485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.765300 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2b296-c832-429b-b630-42858745f485 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.776714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 613/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:23.782887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141d550-024a-4b6b-91e6-a82e520be6d0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:23.792059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:23.880985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8141d550-024a-4b6b-91e6-a82e520be6d0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:23.913716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cbd47-b65d-452e-8c25-21af25837aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:23.914540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cbd47-b65d-452e-8c25-21af25837aa0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:23.914827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cbd47-b65d-452e-8c25-21af25837aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:23.915109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cbd47-b65d-452e-8c25-21af25837aa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:23.921734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cbd47-b65d-452e-8c25-21af25837aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:23.923206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cbd47-b65d-452e-8c25-21af25837aa0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-bd3cbd47-b65d-452e-8c25-21af25837aa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:38:23.937752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:23.938818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:23.940285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:38:23.940285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:38:23.940285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:38:23.940285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:38:23.940285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:38:23.940285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:38:23.940285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:23.940285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:23.940285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 617/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:23.945074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2b296-c832-429b-b630-42858745f485 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:24.048539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2b296-c832-429b-b630-42858745f485 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:24.082923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267e7d4-9282-446f-901a-db88b74051bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:24.083582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267e7d4-9282-446f-901a-db88b74051bb None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:24.083869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267e7d4-9282-446f-901a-db88b74051bb None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:24.084150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267e7d4-9282-446f-901a-db88b74051bb None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:24.084941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267e7d4-9282-446f-901a-db88b74051bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:24.085253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267e7d4-9282-446f-901a-db88b74051bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:24.086740 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267e7d4-9282-446f-901a-db88b74051bb None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:24.099616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 614/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:38:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:24.105735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cbd47-b65d-452e-8c25-21af25837aa0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:24.171941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:24.204571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cbd47-b65d-452e-8c25-21af25837aa0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:24.237658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98813ec8-1cc2-46b6-a6bb-75f46fd97b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:24.238560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813ec8-1cc2-46b6-a6bb-75f46fd97b69 None admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:24.238845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813ec8-1cc2-46b6-a6bb-75f46fd97b69 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:24.239130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98813ec8-1cc2-46b6-a6bb-75f46fd97b69 None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/97524081e59f4775b83d82edf737810e/roles/3cb613c2ddca467295425cf3890d1dc3` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:24.252858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98813ec8-1cc2-46b6-a6bb-75f46fd97b69 None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=97524081e59f4775b83d82edf737810e, role_id=3cb613c2ddca467295425cf3890d1dc3)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:24.254333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98813ec8-1cc2-46b6-a6bb-75f46fd97b69 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:24.366060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 618/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:38:24 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/97524081e59f4775b83d82edf737810e/roles/3cb613c2ddca467295425cf3890d1dc3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:26.377697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecedeb-1074-407e-8f49-7f86430be181 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:26.378402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecedeb-1074-407e-8f49-7f86430be181 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:26.378402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecedeb-1074-407e-8f49-7f86430be181 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:26.379601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 615/1233] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:26.384867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe66cc-ceb4-43a7-af8e-f1e326a53020 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:26.389448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe66cc-ceb4-43a7-af8e-f1e326a53020 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:26.389448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe66cc-ceb4-43a7-af8e-f1e326a53020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:26.402058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-3bfe66cc-ceb4-43a7-af8e-f1e326a53020 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:26.403575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3bfe66cc-ceb4-43a7-af8e-f1e326a53020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:26.470370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 619/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:26.493922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecedeb-1074-407e-8f49-7f86430be181 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:26.520428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:26.598605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecedeb-1074-407e-8f49-7f86430be181 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:26.633798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7df29d-c272-4bb1-a050-11a3858aa651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:26.633798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df29d-c272-4bb1-a050-11a3858aa651 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:26.633798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df29d-c272-4bb1-a050-11a3858aa651 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:26.633798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df29d-c272-4bb1-a050-11a3858aa651 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:26.644549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7df29d-c272-4bb1-a050-11a3858aa651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-6a7df29d-c272-4bb1-a050-11a3858aa651 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:38:26.652775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:26.654018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:26.655157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 616/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 14 15:38:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:26.660883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe66cc-ceb4-43a7-af8e-f1e326a53020 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:26.706770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:26.758520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe66cc-ceb4-43a7-af8e-f1e326a53020 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:26.790881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1ca77-c229-4367-9163-80d830a069e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:26.791554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1ca77-c229-4367-9163-80d830a069e4 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:26.791839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1ca77-c229-4367-9163-80d830a069e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:26.792118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1ca77-c229-4367-9163-80d830a069e4 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:26.793513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1ca77-c229-4367-9163-80d830a069e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:26.793848 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1ca77-c229-4367-9163-80d830a069e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:26.795216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1ca77-c229-4367-9163-80d830a069e4 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:26.803497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 620/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 14 15:38:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:38:26.809225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7df29d-c272-4bb1-a050-11a3858aa651 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:26.893649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:26.909571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7df29d-c272-4bb1-a050-11a3858aa651 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:26.942821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392991a-ac78-43b1-9882-936321fb9083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:26.943606 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392991a-ac78-43b1-9882-936321fb9083 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:26.943909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392991a-ac78-43b1-9882-936321fb9083 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:26.944175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392991a-ac78-43b1-9882-936321fb9083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:26.954514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392991a-ac78-43b1-9882-936321fb9083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:26.955959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9392991a-ac78-43b1-9882-936321fb9083 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-9392991a-ac78-43b1-9882-936321fb9083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:26.973221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:26.974287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:26.975261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:26.975261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:26.975261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:26.975261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:26.975261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 617/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:26.980303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1ca77-c229-4367-9163-80d830a069e4 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:27.077805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1ca77-c229-4367-9163-80d830a069e4 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:27.111379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b567c91-fde2-45e8-9669-32f490d99b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:27.112042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b567c91-fde2-45e8-9669-32f490d99b13 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:27.112329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b567c91-fde2-45e8-9669-32f490d99b13 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:27.112608 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b567c91-fde2-45e8-9669-32f490d99b13 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:27.113333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b567c91-fde2-45e8-9669-32f490d99b13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:27.113644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b567c91-fde2-45e8-9669-32f490d99b13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:27.115038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b567c91-fde2-45e8-9669-32f490d99b13 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:27.125973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 621/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:27.132027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392991a-ac78-43b1-9882-936321fb9083 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:27.160881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:27.231941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392991a-ac78-43b1-9882-936321fb9083 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:27.265042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225433dd-e240-411a-a775-65c224e130ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:27.265861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225433dd-e240-411a-a775-65c224e130ad None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:27.266152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225433dd-e240-411a-a775-65c224e130ad None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:27.266450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225433dd-e240-411a-a775-65c224e130ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:27.274089 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225433dd-e240-411a-a775-65c224e130ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:27.275533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225433dd-e240-411a-a775-65c224e130ad None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-225433dd-e240-411a-a775-65c224e130ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:38:27.290426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:27.291578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:27.292875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:27.293622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 618/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:27.293622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:27.298642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b567c91-fde2-45e8-9669-32f490d99b13 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:27.395476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b567c91-fde2-45e8-9669-32f490d99b13 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:27.429050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a381a-bfaa-43de-90bc-66c4fce83a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:27.429656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a381a-bfaa-43de-90bc-66c4fce83a8a None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:27.429972 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a381a-bfaa-43de-90bc-66c4fce83a8a None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:27.430245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a381a-bfaa-43de-90bc-66c4fce83a8a None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:27.431068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a381a-bfaa-43de-90bc-66c4fce83a8a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:27.431371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a381a-bfaa-43de-90bc-66c4fce83a8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:27.432776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a381a-bfaa-43de-90bc-66c4fce83a8a None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:27.450281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 622/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 14 15:38:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:27.456294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225433dd-e240-411a-a775-65c224e130ad None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:27.555575 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225433dd-e240-411a-a775-65c224e130ad None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:27.589447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef478db-f250-4bc3-939f-2542b9baac93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:27.590160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef478db-f250-4bc3-939f-2542b9baac93 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:27.590449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef478db-f250-4bc3-939f-2542b9baac93 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:27.590731 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef478db-f250-4bc3-939f-2542b9baac93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:27.650901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:27.658166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef478db-f250-4bc3-939f-2542b9baac93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:27.659613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef478db-f250-4bc3-939f-2542b9baac93 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:27.757267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 619/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:38:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:38:27.763080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a381a-bfaa-43de-90bc-66c4fce83a8a None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:27.790369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:27.861179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a381a-bfaa-43de-90bc-66c4fce83a8a None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:27.894782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3c542-0312-41c0-b7cb-772b1ac93a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:27.895574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3c542-0312-41c0-b7cb-772b1ac93a17 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:27.895858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3c542-0312-41c0-b7cb-772b1ac93a17 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:27.896135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3c542-0312-41c0-b7cb-772b1ac93a17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:27.906190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3c542-0312-41c0-b7cb-772b1ac93a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:27.910250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3c542-0312-41c0-b7cb-772b1ac93a17 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-78f3c542-0312-41c0-b7cb-772b1ac93a17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:27.925518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:27.926682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:27.927760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:27.927760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:27.927760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:27.927760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:27.927760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 623/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:38:27.932483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef478db-f250-4bc3-939f-2542b9baac93 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:28.024958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:28.030645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef478db-f250-4bc3-939f-2542b9baac93 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:28.065296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03735a0-b633-46b8-b720-3f0a7a6ec05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:28.065999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03735a0-b633-46b8-b720-3f0a7a6ec05e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:28.070651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03735a0-b633-46b8-b720-3f0a7a6ec05e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:28.070651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03735a0-b633-46b8-b720-3f0a7a6ec05e None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:28.070651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03735a0-b633-46b8-b720-3f0a7a6ec05e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:28.070651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03735a0-b633-46b8-b720-3f0a7a6ec05e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:28.070651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03735a0-b633-46b8-b720-3f0a7a6ec05e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:28.080147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 620/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:28.085940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3c542-0312-41c0-b7cb-772b1ac93a17 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:28.162838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:28.183096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3c542-0312-41c0-b7cb-772b1ac93a17 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:28.216286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fb476-dc67-44bf-8039-02772f6be3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:28.217069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fb476-dc67-44bf-8039-02772f6be3ad None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:28.217348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fb476-dc67-44bf-8039-02772f6be3ad None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:28.217622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fb476-dc67-44bf-8039-02772f6be3ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:28.224044 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fb476-dc67-44bf-8039-02772f6be3ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:28.225521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fb476-dc67-44bf-8039-02772f6be3ad None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-7f0fb476-dc67-44bf-8039-02772f6be3ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:38:28.239001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:28.240084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:28.241223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:38:28.241223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:38:28.241223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:38:28.241223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:38:28.241223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:38:28.241223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:38:28.241223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:28.241223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:28.241223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 624/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:38:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:28.246630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03735a0-b633-46b8-b720-3f0a7a6ec05e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:28.348859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03735a0-b633-46b8-b720-3f0a7a6ec05e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:28.381631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50416da4-324c-4b16-9afb-68a2c2dad71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:28.382312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50416da4-324c-4b16-9afb-68a2c2dad71d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:28.391831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50416da4-324c-4b16-9afb-68a2c2dad71d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:28.391831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50416da4-324c-4b16-9afb-68a2c2dad71d None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:28.391831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50416da4-324c-4b16-9afb-68a2c2dad71d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:28.391831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50416da4-324c-4b16-9afb-68a2c2dad71d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:28.391831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50416da4-324c-4b16-9afb-68a2c2dad71d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:28.397139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 621/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:28.403791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fb476-dc67-44bf-8039-02772f6be3ad None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:28.501582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fb476-dc67-44bf-8039-02772f6be3ad None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:28.533774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8d23d-f5be-4db9-b784-d341b024da0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:28.534450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8d23d-f5be-4db9-b784-d341b024da0c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:28.534731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8d23d-f5be-4db9-b784-d341b024da0c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:28.535005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8d23d-f5be-4db9-b784-d341b024da0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:28.535823 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8d23d-f5be-4db9-b784-d341b024da0c None admin] RBAC: Adding query filter params (role.id=3cb613c2ddca467295425cf3890d1dc3, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=97524081e59f4775b83d82edf737810e) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:28.536128 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8d23d-f5be-4db9-b784-d341b024da0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:28.537545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8d23d-f5be-4db9-b784-d341b024da0c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:28.543810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 625/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:38:28 2022] GET /identity/v3/role_assignments?user.id=97524081e59f4775b83d82edf737810e&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=3cb613c2ddca467295425cf3890d1dc3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:38:30.610204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaeece4-eb78-455d-8ed3-4e0fe8c65f5b None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:30.610733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaeece4-eb78-455d-8ed3-4e0fe8c65f5b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:30.610845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaeece4-eb78-455d-8ed3-4e0fe8c65f5b None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:30.612238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 622/1247] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:30.617576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ebb62-6506-4b7d-acfa-d823809cdb7e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:30.618006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ebb62-6506-4b7d-acfa-d823809cdb7e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:30.618173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ebb62-6506-4b7d-acfa-d823809cdb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:30.623275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:30.635095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-6b8ebb62-6506-4b7d-acfa-d823809cdb7e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:30.636285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6b8ebb62-6506-4b7d-acfa-d823809cdb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:30.701369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 626/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:30.725205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaeece4-eb78-455d-8ed3-4e0fe8c65f5b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:30.743797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:30.825542 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaeece4-eb78-455d-8ed3-4e0fe8c65f5b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:30.859997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bbc6d-324c-4a2b-b4bd-dbdd5f6450db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:30.860837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bbc6d-324c-4a2b-b4bd-dbdd5f6450db None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:30.861094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bbc6d-324c-4a2b-b4bd-dbdd5f6450db None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:30.861368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bbc6d-324c-4a2b-b4bd-dbdd5f6450db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:30.871689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bbc6d-324c-4a2b-b4bd-dbdd5f6450db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-8a2bbc6d-324c-4a2b-b4bd-dbdd5f6450db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:38:30.879901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:30.880999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:30.881891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 623/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:38:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:38:30.888433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ebb62-6506-4b7d-acfa-d823809cdb7e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:30.986086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ebb62-6506-4b7d-acfa-d823809cdb7e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.016027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:31.020826 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6c203-f8a9-4a61-bb3a-3203846438cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.021492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6c203-f8a9-4a61-bb3a-3203846438cc None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.021812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6c203-f8a9-4a61-bb3a-3203846438cc None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.022096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6c203-f8a9-4a61-bb3a-3203846438cc None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.022891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6c203-f8a9-4a61-bb3a-3203846438cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:31.023192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6c203-f8a9-4a61-bb3a-3203846438cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.024553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6c203-f8a9-4a61-bb3a-3203846438cc None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.033204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 627/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:38:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:31.039339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bbc6d-324c-4a2b-b4bd-dbdd5f6450db None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.123994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:31.146775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bbc6d-324c-4a2b-b4bd-dbdd5f6450db None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.179543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90fbfb-4396-464a-b308-888eb37da282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.180335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90fbfb-4396-464a-b308-888eb37da282 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.180647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90fbfb-4396-464a-b308-888eb37da282 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.180906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90fbfb-4396-464a-b308-888eb37da282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.191701 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90fbfb-4396-464a-b308-888eb37da282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.193154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90fbfb-4396-464a-b308-888eb37da282 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-6b90fbfb-4396-464a-b308-888eb37da282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:31.210319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:31.211283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:31.212276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:31.212276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:31.212276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:31.212276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:31.212276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 624/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:38:31.217474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6c203-f8a9-4a61-bb3a-3203846438cc None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.318463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6c203-f8a9-4a61-bb3a-3203846438cc None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.351032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cc810-9794-4491-8045-a6d9c0afe615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.352345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cc810-9794-4491-8045-a6d9c0afe615 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.352345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cc810-9794-4491-8045-a6d9c0afe615 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.352345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cc810-9794-4491-8045-a6d9c0afe615 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.352938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cc810-9794-4491-8045-a6d9c0afe615 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:31.353249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cc810-9794-4491-8045-a6d9c0afe615 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.354629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cc810-9794-4491-8045-a6d9c0afe615 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.366083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 628/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:31.372419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90fbfb-4396-464a-b308-888eb37da282 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.472890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90fbfb-4396-464a-b308-888eb37da282 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.499114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:31.506640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8222b-da07-4b51-b593-161d0fda018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.507431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8222b-da07-4b51-b593-161d0fda018e None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.507748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8222b-da07-4b51-b593-161d0fda018e None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.508022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8222b-da07-4b51-b593-161d0fda018e None admin] PATH_INFO: `/v3/users/placement` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.515611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8222b-da07-4b51-b593-161d0fda018e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.517052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8222b-da07-4b51-b593-161d0fda018e None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-84e8222b-da07-4b51-b593-161d0fda018e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:38:31.531756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:31.532973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:31.534275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 625/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:31.539107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cc810-9794-4491-8045-a6d9c0afe615 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.594771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:31.639041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cc810-9794-4491-8045-a6d9c0afe615 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.672030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27917471-f557-47be-8f2a-17e7b6a7ba64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.672715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27917471-f557-47be-8f2a-17e7b6a7ba64 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.673002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27917471-f557-47be-8f2a-17e7b6a7ba64 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.673277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27917471-f557-47be-8f2a-17e7b6a7ba64 None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.674141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27917471-f557-47be-8f2a-17e7b6a7ba64 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:31.674451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27917471-f557-47be-8f2a-17e7b6a7ba64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.675870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27917471-f557-47be-8f2a-17e7b6a7ba64 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.693144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 629/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 14 15:38:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:31.699417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8222b-da07-4b51-b593-161d0fda018e None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:31.797441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8222b-da07-4b51-b593-161d0fda018e None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:31.830838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ab3bc-b7eb-4090-a4ab-862b6a484779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:31.831483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ab3bc-b7eb-4090-a4ab-862b6a484779 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:31.831770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ab3bc-b7eb-4090-a4ab-862b6a484779 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:31.832047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ab3bc-b7eb-4090-a4ab-862b6a484779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:31.885999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:31.898720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ab3bc-b7eb-4090-a4ab-862b6a484779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:31.900177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ab3bc-b7eb-4090-a4ab-862b6a484779 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:31.997826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 626/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:38:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:38:32.003664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27917471-f557-47be-8f2a-17e7b6a7ba64 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.102643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27917471-f557-47be-8f2a-17e7b6a7ba64 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.115348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:32.136027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfadae75-e7d3-4904-95cb-cc0ab4c0eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.141915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadae75-e7d3-4904-95cb-cc0ab4c0eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.141915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadae75-e7d3-4904-95cb-cc0ab4c0eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.141915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadae75-e7d3-4904-95cb-cc0ab4c0eaf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.147630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfadae75-e7d3-4904-95cb-cc0ab4c0eaf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.149070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfadae75-e7d3-4904-95cb-cc0ab4c0eaf8 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-dfadae75-e7d3-4904-95cb-cc0ab4c0eaf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:32.166080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:32.167178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:32.168141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:32.168141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:32.168141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:32.168141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:32.168141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 630/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:32.173010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ab3bc-b7eb-4090-a4ab-862b6a484779 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.259668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:32.271743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ab3bc-b7eb-4090-a4ab-862b6a484779 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.304054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fbf1d-8b24-409e-86da-202e2102bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.304705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fbf1d-8b24-409e-86da-202e2102bed8 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.304987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fbf1d-8b24-409e-86da-202e2102bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.305262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fbf1d-8b24-409e-86da-202e2102bed8 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.306020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fbf1d-8b24-409e-86da-202e2102bed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:32.306326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fbf1d-8b24-409e-86da-202e2102bed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.314242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fbf1d-8b24-409e-86da-202e2102bed8 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.320016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 627/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:32.326165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfadae75-e7d3-4904-95cb-cc0ab4c0eaf8 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.425227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfadae75-e7d3-4904-95cb-cc0ab4c0eaf8 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.458999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186b849-dc2d-4601-90ce-e52ebc586835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.459767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186b849-dc2d-4601-90ce-e52ebc586835 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.460049 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186b849-dc2d-4601-90ce-e52ebc586835 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.460321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186b849-dc2d-4601-90ce-e52ebc586835 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.467054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186b849-dc2d-4601-90ce-e52ebc586835 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.468486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186b849-dc2d-4601-90ce-e52ebc586835 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-8186b849-dc2d-4601-90ce-e52ebc586835 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:38:32.482489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:32.483631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:32.484838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:38:32.484838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:38:32.484838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:38:32.484838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:38:32.484838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:38:32.484838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:38:32.484838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:32.484838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:32.484838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 631/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:32.489530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fbf1d-8b24-409e-86da-202e2102bed8 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.588738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fbf1d-8b24-409e-86da-202e2102bed8 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.623625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b812bd-3354-47e3-85f9-939817e51b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.624372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b812bd-3354-47e3-85f9-939817e51b7d None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.624718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b812bd-3354-47e3-85f9-939817e51b7d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.625056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b812bd-3354-47e3-85f9-939817e51b7d None admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.625943 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b812bd-3354-47e3-85f9-939817e51b7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:32.626254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b812bd-3354-47e3-85f9-939817e51b7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.627815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b812bd-3354-47e3-85f9-939817e51b7d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.640410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 628/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:32.646547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186b849-dc2d-4601-90ce-e52ebc586835 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:32.672321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:32.743554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186b849-dc2d-4601-90ce-e52ebc586835 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:32.778934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e488bf-4c0b-4712-9676-49aa6ecc4591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:32.779588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e488bf-4c0b-4712-9676-49aa6ecc4591 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:32.779931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e488bf-4c0b-4712-9676-49aa6ecc4591 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:32.780211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e488bf-4c0b-4712-9676-49aa6ecc4591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:32.781044 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e488bf-4c0b-4712-9676-49aa6ecc4591 None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=97524081e59f4775b83d82edf737810e) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:32.781356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e488bf-4c0b-4712-9676-49aa6ecc4591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:32.782809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e488bf-4c0b-4712-9676-49aa6ecc4591 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:32.788200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 632/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:38:32 2022] GET /identity/v3/role_assignments?user.id=97524081e59f4775b83d82edf737810e&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=abfb60dccca7470295258595a66a5d17 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:34.789386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ab76-98e1-43e7-844c-a114b158e10b None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:34.789959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ab76-98e1-43e7-844c-a114b158e10b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:34.789959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ab76-98e1-43e7-844c-a114b158e10b None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:34.791354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 629/1261] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:34.797160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4718568-ac88-490e-8264-0a544204544c None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:34.804416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4718568-ac88-490e-8264-0a544204544c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:34.804416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4718568-ac88-490e-8264-0a544204544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:34.815264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d4718568-ac88-490e-8264-0a544204544c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:34.816492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d4718568-ac88-490e-8264-0a544204544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:34.884531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 633/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:34.906699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6ab76-98e1-43e7-844c-a114b158e10b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:34.918101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:35.009082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6ab76-98e1-43e7-844c-a114b158e10b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:35.043553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b714fcb-dda6-41b1-abc1-07dd6a503068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:35.044345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b714fcb-dda6-41b1-abc1-07dd6a503068 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:35.044666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b714fcb-dda6-41b1-abc1-07dd6a503068 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:35.044940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b714fcb-dda6-41b1-abc1-07dd6a503068 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:35.055132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b714fcb-dda6-41b1-abc1-07dd6a503068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-4b714fcb-dda6-41b1-abc1-07dd6a503068 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:38:35.062812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:35.064202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:35.065441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 630/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:38:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:35.070387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4718568-ac88-490e-8264-0a544204544c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:35.089076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:35.169952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4718568-ac88-490e-8264-0a544204544c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:35.203780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16723bb0-200d-4949-8849-33b7a4e15ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:35.204450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16723bb0-200d-4949-8849-33b7a4e15ef0 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:35.204735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16723bb0-200d-4949-8849-33b7a4e15ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:35.205013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16723bb0-200d-4949-8849-33b7a4e15ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:35.205758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16723bb0-200d-4949-8849-33b7a4e15ef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:35.206097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16723bb0-200d-4949-8849-33b7a4e15ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:35.207455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16723bb0-200d-4949-8849-33b7a4e15ef0 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:35.215406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 634/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:38:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:35.221337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b714fcb-dda6-41b1-abc1-07dd6a503068 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:35.287732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:35.321570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b714fcb-dda6-41b1-abc1-07dd6a503068 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:35.354688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfc547-593a-4c3d-a280-c8d908541a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:35.355501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfc547-593a-4c3d-a280-c8d908541a5a None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:35.355780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfc547-593a-4c3d-a280-c8d908541a5a None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:35.356050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfc547-593a-4c3d-a280-c8d908541a5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:35.366408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfc547-593a-4c3d-a280-c8d908541a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:35.367883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfc547-593a-4c3d-a280-c8d908541a5a None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-44cfc547-593a-4c3d-a280-c8d908541a5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:35.384878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:35.385954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:35.387565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:35.387565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:35.387565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:35.387565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:35.387565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 631/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:35.392908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16723bb0-200d-4949-8849-33b7a4e15ef0 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:35.475246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:35.494220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16723bb0-200d-4949-8849-33b7a4e15ef0 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:35.526925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bf1f9-1234-494d-aaee-fd092415caa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:35.527591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bf1f9-1234-494d-aaee-fd092415caa9 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:35.527878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bf1f9-1234-494d-aaee-fd092415caa9 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:35.528157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bf1f9-1234-494d-aaee-fd092415caa9 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:35.528874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bf1f9-1234-494d-aaee-fd092415caa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:35.529192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bf1f9-1234-494d-aaee-fd092415caa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:35.530571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bf1f9-1234-494d-aaee-fd092415caa9 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:35.541993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 635/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:35.548432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfc547-593a-4c3d-a280-c8d908541a5a None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:35.648410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfc547-593a-4c3d-a280-c8d908541a5a None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:35.668821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:35.682830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2aa02-482f-4900-8e9b-4bc3860b58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:35.684182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2aa02-482f-4900-8e9b-4bc3860b58c9 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:35.684467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2aa02-482f-4900-8e9b-4bc3860b58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:35.684747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2aa02-482f-4900-8e9b-4bc3860b58c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:35.692823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2aa02-482f-4900-8e9b-4bc3860b58c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:35.694469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2aa02-482f-4900-8e9b-4bc3860b58c9 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-94f2aa02-482f-4900-8e9b-4bc3860b58c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:38:35.708600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:35.709730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:35.715673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 632/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:35.716320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bf1f9-1234-494d-aaee-fd092415caa9 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:35.815978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bf1f9-1234-494d-aaee-fd092415caa9 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:35.850422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bad9ec-3f4d-4d55-9855-820a317cf39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:35.851145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bad9ec-3f4d-4d55-9855-820a317cf39f None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:35.851427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bad9ec-3f4d-4d55-9855-820a317cf39f None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:35.851704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bad9ec-3f4d-4d55-9855-820a317cf39f None admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:35.852535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bad9ec-3f4d-4d55-9855-820a317cf39f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:35.852846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bad9ec-3f4d-4d55-9855-820a317cf39f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:35.854345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bad9ec-3f4d-4d55-9855-820a317cf39f None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:35.863658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:35.871909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 636/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 14 15:38:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:35.877751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2aa02-482f-4900-8e9b-4bc3860b58c9 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:35.976257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2aa02-482f-4900-8e9b-4bc3860b58c9 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.009555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18840-75e2-445f-81cf-715100542983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.010365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18840-75e2-445f-81cf-715100542983 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.010652 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18840-75e2-445f-81cf-715100542983 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.010930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf18840-75e2-445f-81cf-715100542983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.021394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf18840-75e2-445f-81cf-715100542983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.022891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf18840-75e2-445f-81cf-715100542983 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-bbf18840-75e2-445f-81cf-715100542983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:36.040268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:36.041353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:36.051817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:36.051817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:36.051817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:36.051817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:36.051817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 633/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:36.052694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bad9ec-3f4d-4d55-9855-820a317cf39f None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:36.144809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bad9ec-3f4d-4d55-9855-820a317cf39f None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.178532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca934c-768a-4249-816d-3965f98581e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.180460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca934c-768a-4249-816d-3965f98581e5 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.180460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca934c-768a-4249-816d-3965f98581e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.180460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca934c-768a-4249-816d-3965f98581e5 None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.184489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca934c-768a-4249-816d-3965f98581e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:36.185039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca934c-768a-4249-816d-3965f98581e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.186638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca934c-768a-4249-816d-3965f98581e5 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.200080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 637/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:36.205763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18840-75e2-445f-81cf-715100542983 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:36.238606 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:36.304926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf18840-75e2-445f-81cf-715100542983 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.337866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778d752-b521-4b6a-8f56-b63e785c72d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.338647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778d752-b521-4b6a-8f56-b63e785c72d7 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.338966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778d752-b521-4b6a-8f56-b63e785c72d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.339293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778d752-b521-4b6a-8f56-b63e785c72d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.346696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778d752-b521-4b6a-8f56-b63e785c72d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.348122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778d752-b521-4b6a-8f56-b63e785c72d7 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-a778d752-b521-4b6a-8f56-b63e785c72d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:38:36.362641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:36.363730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:36.365038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:38:36.365038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:38:36.365038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:38:36.365038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:38:36.365038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:38:36.365038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:38:36.365038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:36.365038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:36.365038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 634/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:36.369674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca934c-768a-4249-816d-3965f98581e5 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:36.427700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:36.467425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca934c-768a-4249-816d-3965f98581e5 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.500983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ebb92-7fa7-4390-861e-78649b870d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.501591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ebb92-7fa7-4390-861e-78649b870d3e None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.501906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ebb92-7fa7-4390-861e-78649b870d3e None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.502187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ebb92-7fa7-4390-861e-78649b870d3e None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.502977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ebb92-7fa7-4390-861e-78649b870d3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:36.503311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ebb92-7fa7-4390-861e-78649b870d3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.504737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ebb92-7fa7-4390-861e-78649b870d3e None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.516644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 638/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:36.522626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778d752-b521-4b6a-8f56-b63e785c72d7 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:36.614090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:36.622715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778d752-b521-4b6a-8f56-b63e785c72d7 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:36.656976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4ce62-193c-4014-ac81-34566c06191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:36.657885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4ce62-193c-4014-ac81-34566c06191d None admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:36.658168 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4ce62-193c-4014-ac81-34566c06191d None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:36.658447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4ce62-193c-4014-ac81-34566c06191d None admin] PATH_INFO: `/v3/projects/3056a2f260314184a08f3a9101a335d2/users/97524081e59f4775b83d82edf737810e/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:36.660366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4ce62-193c-4014-ac81-34566c06191d None admin] RBAC: Authorizing `identity:create_grant(project_id=3056a2f260314184a08f3a9101a335d2, user_id=97524081e59f4775b83d82edf737810e, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:36.661793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4ce62-193c-4014-ac81-34566c06191d None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:36.795532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 635/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 14 15:38:36 2022] PUT /identity/v3/projects/3056a2f260314184a08f3a9101a335d2/users/97524081e59f4775b83d82edf737810e/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:38:38.789007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939122a3-8b1c-4ac3-9b4f-da531a14769d None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.789557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939122a3-8b1c-4ac3-9b4f-da531a14769d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.789557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939122a3-8b1c-4ac3-9b4f-da531a14769d None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.791023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 639/1274] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:38.796225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b6c6-236f-4676-912a-20503a0526d0 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:38.796625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b6c6-236f-4676-912a-20503a0526d0 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:38.796782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b6c6-236f-4676-912a-20503a0526d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:38.813288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-8ed8b6c6-236f-4676-912a-20503a0526d0 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:38.814773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8ed8b6c6-236f-4676-912a-20503a0526d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:38.881109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 636/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:38.904404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939122a3-8b1c-4ac3-9b4f-da531a14769d None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.005012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939122a3-8b1c-4ac3-9b4f-da531a14769d None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.022478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:39.040741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610242e6-3f7c-4689-81da-920c8893f841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.041480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610242e6-3f7c-4689-81da-920c8893f841 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.041761 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610242e6-3f7c-4689-81da-920c8893f841 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.042506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610242e6-3f7c-4689-81da-920c8893f841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.052365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610242e6-3f7c-4689-81da-920c8893f841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-610242e6-3f7c-4689-81da-920c8893f841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 15:38:39.060525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:39.061685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:39.062874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 640/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 14 15:38:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:39.068022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8b6c6-236f-4676-912a-20503a0526d0 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.149011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:39.167123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8b6c6-236f-4676-912a-20503a0526d0 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.208046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87a72f-6b31-4a59-858f-4e2d7b8bfc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.208717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87a72f-6b31-4a59-858f-4e2d7b8bfc89 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.209001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87a72f-6b31-4a59-858f-4e2d7b8bfc89 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.209278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87a72f-6b31-4a59-858f-4e2d7b8bfc89 None admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.210063 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87a72f-6b31-4a59-858f-4e2d7b8bfc89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:39.210375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87a72f-6b31-4a59-858f-4e2d7b8bfc89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.211748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87a72f-6b31-4a59-858f-4e2d7b8bfc89 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.220283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 637/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:39.226046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610242e6-3f7c-4689-81da-920c8893f841 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.325418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610242e6-3f7c-4689-81da-920c8893f841 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.357205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445494f6-6a63-4063-86c7-389d0efe879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.358011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445494f6-6a63-4063-86c7-389d0efe879c None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.358292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445494f6-6a63-4063-86c7-389d0efe879c None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.358563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445494f6-6a63-4063-86c7-389d0efe879c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.368651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445494f6-6a63-4063-86c7-389d0efe879c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.370094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445494f6-6a63-4063-86c7-389d0efe879c None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-445494f6-6a63-4063-86c7-389d0efe879c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:39.387131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:39.388279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:39.389547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:39.389547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:39.389547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:39.389547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:39.389547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 641/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:39.394084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87a72f-6b31-4a59-858f-4e2d7b8bfc89 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.496473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87a72f-6b31-4a59-858f-4e2d7b8bfc89 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.529514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdba02-d8a9-496e-bec1-4df26dac4cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.530205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdba02-d8a9-496e-bec1-4df26dac4cd6 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.530489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdba02-d8a9-496e-bec1-4df26dac4cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.530765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdba02-d8a9-496e-bec1-4df26dac4cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.531636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdba02-d8a9-496e-bec1-4df26dac4cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:39.531943 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdba02-d8a9-496e-bec1-4df26dac4cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.533310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdba02-d8a9-496e-bec1-4df26dac4cd6 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.545565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 638/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:39.551367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445494f6-6a63-4063-86c7-389d0efe879c None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.582685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:39.649430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445494f6-6a63-4063-86c7-389d0efe879c None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.682665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad6e39-2332-4148-ba0f-730731d049ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.683568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad6e39-2332-4148-ba0f-730731d049ee None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.683854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad6e39-2332-4148-ba0f-730731d049ee None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.684135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad6e39-2332-4148-ba0f-730731d049ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.691882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad6e39-2332-4148-ba0f-730731d049ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.693316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad6e39-2332-4148-ba0f-730731d049ee None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-dfad6e39-2332-4148-ba0f-730731d049ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 15:38:39.707276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:39.709317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:39.710470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 642/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:39.715052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdba02-d8a9-496e-bec1-4df26dac4cd6 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.718808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:39.814491 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdba02-d8a9-496e-bec1-4df26dac4cd6 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:39.848377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91e3ee-752f-4d8f-a522-b8a14e683df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:39.848992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91e3ee-752f-4d8f-a522-b8a14e683df4 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:39.849275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91e3ee-752f-4d8f-a522-b8a14e683df4 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:39.849549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91e3ee-752f-4d8f-a522-b8a14e683df4 None admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:39.850418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91e3ee-752f-4d8f-a522-b8a14e683df4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:39.850725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91e3ee-752f-4d8f-a522-b8a14e683df4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:39.852223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91e3ee-752f-4d8f-a522-b8a14e683df4 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:39.869733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 639/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:39.875760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad6e39-2332-4148-ba0f-730731d049ee None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:39.968419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:39.977888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad6e39-2332-4148-ba0f-730731d049ee None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.010181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0713d1b-9646-4f70-b2bd-9456b0a55760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.010855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0713d1b-9646-4f70-b2bd-9456b0a55760 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.011136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0713d1b-9646-4f70-b2bd-9456b0a55760 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.011968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0713d1b-9646-4f70-b2bd-9456b0a55760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.079941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0713d1b-9646-4f70-b2bd-9456b0a55760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.081395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0713d1b-9646-4f70-b2bd-9456b0a55760 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.167141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:40.176904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 643/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 14 15:38:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 15:38:40.182844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91e3ee-752f-4d8f-a522-b8a14e683df4 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.243993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:40.282937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91e3ee-752f-4d8f-a522-b8a14e683df4 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.317182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470d377-1741-4f4e-add9-0cb406d1ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.318147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470d377-1741-4f4e-add9-0cb406d1ea47 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.318491 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470d377-1741-4f4e-add9-0cb406d1ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.318762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470d377-1741-4f4e-add9-0cb406d1ea47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.329268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470d377-1741-4f4e-add9-0cb406d1ea47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.330833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470d377-1741-4f4e-add9-0cb406d1ea47 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-5470d377-1741-4f4e-add9-0cb406d1ea47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 15:38:40.348468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:40.349574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:40.350551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 15:38:40.350551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 15:38:40.350551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 15:38:40.350551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:40.350551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 640/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:40.355526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0713d1b-9646-4f70-b2bd-9456b0a55760 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.453704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0713d1b-9646-4f70-b2bd-9456b0a55760 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.486633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aa977-03f4-4b9d-beba-02db63da82fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.487293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aa977-03f4-4b9d-beba-02db63da82fe None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.487570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aa977-03f4-4b9d-beba-02db63da82fe None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.487850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931aa977-03f4-4b9d-beba-02db63da82fe None admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.488615 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931aa977-03f4-4b9d-beba-02db63da82fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:40.488961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931aa977-03f4-4b9d-beba-02db63da82fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.490343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931aa977-03f4-4b9d-beba-02db63da82fe None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.502288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 644/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:40.508085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470d377-1741-4f4e-add9-0cb406d1ea47 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.610124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470d377-1741-4f4e-add9-0cb406d1ea47 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.624728 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:40.645332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fc70e-7894-430c-a95d-8f9aed86cdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.646170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fc70e-7894-430c-a95d-8f9aed86cdb1 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.646463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fc70e-7894-430c-a95d-8f9aed86cdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.646739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fc70e-7894-430c-a95d-8f9aed86cdb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.653948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fc70e-7894-430c-a95d-8f9aed86cdb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.655437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fc70e-7894-430c-a95d-8f9aed86cdb1 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application [None req-a50fc70e-7894-430c-a95d-8f9aed86cdb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 15:38:40.670554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:40.671638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:40.672613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 15:38:40.672613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 15:38:40.672613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 15:38:40.672613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 15:38:40.672613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 15:38:40.672613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 15:38:40.672613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 15:38:40.672613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: ERROR keystone.server.flask.application Jul 14 15:38:40.672613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 641/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 15:38:40.677614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aa977-03f4-4b9d-beba-02db63da82fe None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.728246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:40.776710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931aa977-03f4-4b9d-beba-02db63da82fe None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.809137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dee474-2bd0-440b-a71a-f091534dceff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.809795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dee474-2bd0-440b-a71a-f091534dceff None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.810111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dee474-2bd0-440b-a71a-f091534dceff None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.810390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dee474-2bd0-440b-a71a-f091534dceff None admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.811168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dee474-2bd0-440b-a71a-f091534dceff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:40.811479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dee474-2bd0-440b-a71a-f091534dceff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.812933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dee474-2bd0-440b-a71a-f091534dceff None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.825854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 645/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:40.832044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fc70e-7894-430c-a95d-8f9aed86cdb1 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:40.930822 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fc70e-7894-430c-a95d-8f9aed86cdb1 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:40.964671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6a76e-c5a6-4dff-927c-e6040f83abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:40.965346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6a76e-c5a6-4dff-927c-e6040f83abeb None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:40.965630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6a76e-c5a6-4dff-927c-e6040f83abeb None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:40.965939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6a76e-c5a6-4dff-927c-e6040f83abeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:40.966765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6a76e-c5a6-4dff-927c-e6040f83abeb None admin] RBAC: Adding query filter params (role.id=abfb60dccca7470295258595a66a5d17, scope.project.id=3056a2f260314184a08f3a9101a335d2, user.id=97524081e59f4775b83d82edf737810e) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:40.967076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6a76e-c5a6-4dff-927c-e6040f83abeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:40.968506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6a76e-c5a6-4dff-927c-e6040f83abeb None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:40.974425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 642/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 14 15:38:40 2022] GET /identity/v3/role_assignments?user.id=97524081e59f4775b83d82edf737810e&scope.project.id=3056a2f260314184a08f3a9101a335d2&role.id=abfb60dccca7470295258595a66a5d17 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:38:43.005263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ae5d8-22ca-49ed-965d-e1c900214a60 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:43.005854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ae5d8-22ca-49ed-965d-e1c900214a60 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:43.005854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ae5d8-22ca-49ed-965d-e1c900214a60 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:43.007202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 646/1288] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:43.013101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98d0fe-100d-4d3c-9153-5019c5bc115c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:43.013531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98d0fe-100d-4d3c-9153-5019c5bc115c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:43.013703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98d0fe-100d-4d3c-9153-5019c5bc115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:43.030879 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-fd98d0fe-100d-4d3c-9153-5019c5bc115c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:43.032106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fd98d0fe-100d-4d3c-9153-5019c5bc115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:43.062013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:43.100804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 643/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:43.122477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ae5d8-22ca-49ed-965d-e1c900214a60 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:43.223771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ae5d8-22ca-49ed-965d-e1c900214a60 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:43.234235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:43.257919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650e969-a8ac-44c3-831c-9043667d3c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:43.258657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650e969-a8ac-44c3-831c-9043667d3c16 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:43.258938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650e969-a8ac-44c3-831c-9043667d3c16 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:43.259216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650e969-a8ac-44c3-831c-9043667d3c16 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:43.259871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650e969-a8ac-44c3-831c-9043667d3c16 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:43.261283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650e969-a8ac-44c3-831c-9043667d3c16 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-4650e969-a8ac-44c3-831c-9043667d3c16 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:43.273398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:43.274746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:43.275819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 647/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:38:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:43.280620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98d0fe-100d-4d3c-9153-5019c5bc115c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:43.378254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98d0fe-100d-4d3c-9153-5019c5bc115c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:43.412096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb4d78-f66f-4a77-84c3-d331dfaf4959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:43.412750 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb4d78-f66f-4a77-84c3-d331dfaf4959 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:43.413034 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb4d78-f66f-4a77-84c3-d331dfaf4959 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:43.413309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb4d78-f66f-4a77-84c3-d331dfaf4959 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:43.414150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb4d78-f66f-4a77-84c3-d331dfaf4959 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:43.414458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb4d78-f66f-4a77-84c3-d331dfaf4959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:43.415831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb4d78-f66f-4a77-84c3-d331dfaf4959 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:43.420617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 644/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jul 14 15:38:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:43.426389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650e969-a8ac-44c3-831c-9043667d3c16 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:43.528250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650e969-a8ac-44c3-831c-9043667d3c16 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:43.562999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a8b60-6225-408f-a7f4-140f4e0665bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:43.563947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a8b60-6225-408f-a7f4-140f4e0665bf None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:43.564222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a8b60-6225-408f-a7f4-140f4e0665bf None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:43.564498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a8b60-6225-408f-a7f4-140f4e0665bf None admin] PATH_INFO: `/v3/services` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:43.565207 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a8b60-6225-408f-a7f4-140f4e0665bf None admin] RBAC: Adding query filter params (type=placement) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:43.565516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a8b60-6225-408f-a7f4-140f4e0665bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:43.566909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a8b60-6225-408f-a7f4-140f4e0665bf None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:43.571321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 648/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jul 14 15:38:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:45.588787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357dde2-9332-4bda-bb62-e4748634f45b None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:45.589900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357dde2-9332-4bda-bb62-e4748634f45b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:45.589900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357dde2-9332-4bda-bb62-e4748634f45b None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:45.591283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 645/1293] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:45.596617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e4238-d270-43f4-a7c7-39ebc3d6397f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:45.597040 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e4238-d270-43f4-a7c7-39ebc3d6397f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:45.600703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e4238-d270-43f4-a7c7-39ebc3d6397f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:45.614257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cb1e4238-d270-43f4-a7c7-39ebc3d6397f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:45.615448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cb1e4238-d270-43f4-a7c7-39ebc3d6397f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:45.644360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:45.681324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 649/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:45.704587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357dde2-9332-4bda-bb62-e4748634f45b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:45.737179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:45.806952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357dde2-9332-4bda-bb62-e4748634f45b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:45.840497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce72db-8644-405c-81d8-a7b40a5b0a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:45.841098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce72db-8644-405c-81d8-a7b40a5b0a38 None admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:45.841380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce72db-8644-405c-81d8-a7b40a5b0a38 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:45.841654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce72db-8644-405c-81d8-a7b40a5b0a38 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:45.842522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce72db-8644-405c-81d8-a7b40a5b0a38 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:45.843888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce72db-8644-405c-81d8-a7b40a5b0a38 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:45.974342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 646/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 14 15:38:45 2022] POST /identity/v3/services => generated 246 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:38:48.005624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1072f2-037e-49ac-8cfa-9dc02e3863d0 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.006221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1072f2-037e-49ac-8cfa-9dc02e3863d0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.006221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1072f2-037e-49ac-8cfa-9dc02e3863d0 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.007675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 650/1296] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:48.013078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaaa65-73b2-4d12-b9c8-e2b0a1c2b0f7 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.013919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaaa65-73b2-4d12-b9c8-e2b0a1c2b0f7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.013919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaaa65-73b2-4d12-b9c8-e2b0a1c2b0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.030826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-99eaaa65-73b2-4d12-b9c8-e2b0a1c2b0f7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:48.032069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-99eaaa65-73b2-4d12-b9c8-e2b0a1c2b0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:48.105263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 647/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:48.128620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1072f2-037e-49ac-8cfa-9dc02e3863d0 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:48.228393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1072f2-037e-49ac-8cfa-9dc02e3863d0 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:48.262130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b8995-7bc0-4975-8942-66a63c720aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:48.262861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b8995-7bc0-4975-8942-66a63c720aea None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.263143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b8995-7bc0-4975-8942-66a63c720aea None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.263416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b8995-7bc0-4975-8942-66a63c720aea None admin] PATH_INFO: `/v3/services/placement` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.264067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b8995-7bc0-4975-8942-66a63c720aea None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:48.265596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b8995-7bc0-4975-8942-66a63c720aea None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-cd0b8995-7bc0-4975-8942-66a63c720aea None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:48.278399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:48.280210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:48.281467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 651/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:38:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:48.285681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eaaa65-73b2-4d12-b9c8-e2b0a1c2b0f7 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:48.304025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:48.382218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eaaa65-73b2-4d12-b9c8-e2b0a1c2b0f7 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:48.421502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a253935-4ac3-4714-a4c8-cf1d380e0108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:48.422233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a253935-4ac3-4714-a4c8-cf1d380e0108 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.422518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a253935-4ac3-4714-a4c8-cf1d380e0108 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.422794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a253935-4ac3-4714-a4c8-cf1d380e0108 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.423521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a253935-4ac3-4714-a4c8-cf1d380e0108 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:48.423828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a253935-4ac3-4714-a4c8-cf1d380e0108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:48.425241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a253935-4ac3-4714-a4c8-cf1d380e0108 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:48.436419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 648/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jul 14 15:38:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:38:48.437749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b8995-7bc0-4975-8942-66a63c720aea None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:48.473369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:48.546494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b8995-7bc0-4975-8942-66a63c720aea None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:48.583771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab3ee2-65f1-4854-b973-ba456065aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:48.584460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab3ee2-65f1-4854-b973-ba456065aac6 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.584749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab3ee2-65f1-4854-b973-ba456065aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.585021 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab3ee2-65f1-4854-b973-ba456065aac6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.585865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab3ee2-65f1-4854-b973-ba456065aac6 None admin] RBAC: Adding query filter params (interface=public, service_id=4edae45827cb48b2ae7399cd65183030, region_id=RegionOne) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:48.586184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab3ee2-65f1-4854-b973-ba456065aac6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:48.587648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab3ee2-65f1-4854-b973-ba456065aac6 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:48.593942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 652/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 14 15:38:48 2022] GET /identity/v3/endpoints?service_id=4edae45827cb48b2ae7399cd65183030&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:48.599921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a253935-4ac3-4714-a4c8-cf1d380e0108 None admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:48.688976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:48.708285 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a253935-4ac3-4714-a4c8-cf1d380e0108 None admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:48.743339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0e647-e9dc-4eb8-ac96-dc0cf4ea81f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:48.744014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0e647-e9dc-4eb8-ac96-dc0cf4ea81f6 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:48.746064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0e647-e9dc-4eb8-ac96-dc0cf4ea81f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:48.746064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0e647-e9dc-4eb8-ac96-dc0cf4ea81f6 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:48.746064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0e647-e9dc-4eb8-ac96-dc0cf4ea81f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:48.747280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0e647-e9dc-4eb8-ac96-dc0cf4ea81f6 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:48.754291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 649/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 14 15:38:48 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:38:50.788024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c85d9f-9e30-4174-b8bd-fdb561f0ab67 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.788607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c85d9f-9e30-4174-b8bd-fdb561f0ab67 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.788607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c85d9f-9e30-4174-b8bd-fdb561f0ab67 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.790015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 653/1302] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:38:50.795746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e201dc-83e8-4b9f-9cb0-0b51c90d1d0f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:50.796190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e201dc-83e8-4b9f-9cb0-0b51c90d1d0f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:50.796299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e201dc-83e8-4b9f-9cb0-0b51c90d1d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:50.812396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e0e201dc-83e8-4b9f-9cb0-0b51c90d1d0f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:38:50.813928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e0e201dc-83e8-4b9f-9cb0-0b51c90d1d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:38:50.880635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 650/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:38:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 15:38:50.903911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c85d9f-9e30-4174-b8bd-fdb561f0ab67 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:50.999686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:51.006412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c85d9f-9e30-4174-b8bd-fdb561f0ab67 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:51.043925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05246d7-68af-4dc2-9e60-d3c98ca17981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:51.044781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05246d7-68af-4dc2-9e60-d3c98ca17981 None admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:51.045059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05246d7-68af-4dc2-9e60-d3c98ca17981 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:51.045330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05246d7-68af-4dc2-9e60-d3c98ca17981 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:51.046015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05246d7-68af-4dc2-9e60-d3c98ca17981 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:51.047370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05246d7-68af-4dc2-9e60-d3c98ca17981 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application [None req-f05246d7-68af-4dc2-9e60-d3c98ca17981 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 15:38:51.059281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 15:38:51.060487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application with Lock( Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self._enter() Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application return self.creator() Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application created_value = creator( Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: ERROR keystone.server.flask.application Jul 14 15:38:51.061552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 654/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 14 15:38:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 15:38:51.067064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e201dc-83e8-4b9f-9cb0-0b51c90d1d0f None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:51.145214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:38:51.168001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e201dc-83e8-4b9f-9cb0-0b51c90d1d0f None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:51.201884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71fe09-9ce4-4a10-9cfd-9c3aa1329d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:51.202545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71fe09-9ce4-4a10-9cfd-9c3aa1329d26 None admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:51.202824 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71fe09-9ce4-4a10-9cfd-9c3aa1329d26 None admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:51.203093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71fe09-9ce4-4a10-9cfd-9c3aa1329d26 None admin] PATH_INFO: `/v3/services` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:51.203792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71fe09-9ce4-4a10-9cfd-9c3aa1329d26 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:38:51.204094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71fe09-9ce4-4a10-9cfd-9c3aa1329d26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:51.205490 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71fe09-9ce4-4a10-9cfd-9c3aa1329d26 None admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:51.210544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 651/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jul 14 15:38:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:38:51.216900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05246d7-68af-4dc2-9e60-d3c98ca17981 None admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:38:51.314390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05246d7-68af-4dc2-9e60-d3c98ca17981 None admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:38:51.347579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e85f5-2bc0-4b3d-aaf1-8c3d71383cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:38:51.348232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e85f5-2bc0-4b3d-aaf1-8c3d71383cd8 None admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:38:51.348513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e85f5-2bc0-4b3d-aaf1-8c3d71383cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:38:51.348781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e85f5-2bc0-4b3d-aaf1-8c3d71383cd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:38:51.349507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e85f5-2bc0-4b3d-aaf1-8c3d71383cd8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:38:51.350895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e85f5-2bc0-4b3d-aaf1-8c3d71383cd8 None admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:38:51.417233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 655/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 14 15:38:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:39:01.207530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=30 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:39:01.412839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:23.298186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6575375-f5a2-422d-8cec-ec842b7c5a85 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:23.299105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6575375-f5a2-422d-8cec-ec842b7c5a85 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:23.299584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6575375-f5a2-422d-8cec-ec842b7c5a85 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:23.307538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4ad1e-ddf2-41fd-a3ae-4577700b3bcb None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:23.307995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4ad1e-ddf2-41fd-a3ae-4577700b3bcb None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:23.308141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4ad1e-ddf2-41fd-a3ae-4577700b3bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:23.311410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 652/1307] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:23.407912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-eec4ad1e-ddf2-41fd-a3ae-4577700b3bcb None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:23.409718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-eec4ad1e-ddf2-41fd-a3ae-4577700b3bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:23.558980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 656/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:23.608417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479fe94-056b-44d6-b6f6-a4323be18cfc None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:23.608706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479fe94-056b-44d6-b6f6-a4323be18cfc None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:23.609230 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479fe94-056b-44d6-b6f6-a4323be18cfc None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:23.610793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 653/1309] 10.222.0.47 () {54 vars in 964 bytes} [Thu Jul 14 15:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:23.617081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a4c42-1894-42bd-99be-f10ebada8ffc None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:23.617373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a4c42-1894-42bd-99be-f10ebada8ffc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:23.617651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a4c42-1894-42bd-99be-f10ebada8ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:23.694404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-5f2a4c42-1894-42bd-99be-f10ebada8ffc None None] MFA Rules not processed for user `7e63b8e907764f2c818391ab2ad9184c`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:23.695599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5f2a4c42-1894-42bd-99be-f10ebada8ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:23.717319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:23.775859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 657/1310] 10.222.0.47 () {58 vars in 1061 bytes} [Thu Jul 14 15:46:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:23.785025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479fe94-056b-44d6-b6f6-a4323be18cfc None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:23.786816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6479fe94-056b-44d6-b6f6-a4323be18cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:23.912466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479fe94-056b-44d6-b6f6-a4323be18cfc None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:23.994932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe6761-846d-4fa0-98cc-75344779c80f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:23.995510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe6761-846d-4fa0-98cc-75344779c80f service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:23.995795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe6761-846d-4fa0-98cc-75344779c80f service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:23.996080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe6761-846d-4fa0-98cc-75344779c80f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:23.997787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-bcfe6761-846d-4fa0-98cc-75344779c80f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:24.061659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:24.117872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe6761-846d-4fa0-98cc-75344779c80f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:24.119354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe6761-846d-4fa0-98cc-75344779c80f service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:24.139128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 654/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:26.482176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f4c89-c1be-4457-986e-d2d57cfc695e None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:26.482730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f4c89-c1be-4457-986e-d2d57cfc695e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:26.482730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f4c89-c1be-4457-986e-d2d57cfc695e None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:26.484284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 658/1312] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:26.489330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bca19a-2fe3-40d3-9752-978d15ed4079 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:26.489723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bca19a-2fe3-40d3-9752-978d15ed4079 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:26.489976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bca19a-2fe3-40d3-9752-978d15ed4079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:26.575409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-98bca19a-2fe3-40d3-9752-978d15ed4079 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:26.576628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-98bca19a-2fe3-40d3-9752-978d15ed4079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:26.678525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 655/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:26.717626 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f4c89-c1be-4457-986e-d2d57cfc695e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:26.719388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a95f4c89-c1be-4457-986e-d2d57cfc695e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:26.838119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f4c89-c1be-4457-986e-d2d57cfc695e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:26.920287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:26.923586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a02761-ce06-423c-a428-c3b63cb4197c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:26.924605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a02761-ce06-423c-a428-c3b63cb4197c service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:26.925008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a02761-ce06-423c-a428-c3b63cb4197c service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:26.925400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a02761-ce06-423c-a428-c3b63cb4197c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:26.927392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e9a02761-ce06-423c-a428-c3b63cb4197c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:27.060537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a02761-ce06-423c-a428-c3b63cb4197c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:27.062050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a02761-ce06-423c-a428-c3b63cb4197c service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:27.082815 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 659/1314] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:29.695148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f72996-bc3b-4c8d-9a9d-afc5382ce263 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:29.695683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f72996-bc3b-4c8d-9a9d-afc5382ce263 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:29.695683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f72996-bc3b-4c8d-9a9d-afc5382ce263 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:29.697667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 656/1315] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:29.702768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7e16d-e952-4a6f-a0ac-acf57b326794 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:29.703198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7e16d-e952-4a6f-a0ac-acf57b326794 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:29.703380 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7e16d-e952-4a6f-a0ac-acf57b326794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:29.767870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f2f7e16d-e952-4a6f-a0ac-acf57b326794 None None] MFA Rules not processed for user `83ad07150d134e288ea59a065b959f79`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:29.769080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f2f7e16d-e952-4a6f-a0ac-acf57b326794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:29.819840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:29.864099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 660/1316] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:29.918728 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110a748-530a-40ed-b259-864956bfc214 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:29.919008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110a748-530a-40ed-b259-864956bfc214 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:29.919281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110a748-530a-40ed-b259-864956bfc214 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:29.920656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 657/1317] 10.222.0.47 () {54 vars in 964 bytes} [Thu Jul 14 15:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:29.926487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149fb9f-94f0-4418-8995-e541e88668c3 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:29.926781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149fb9f-94f0-4418-8995-e541e88668c3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:29.927064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149fb9f-94f0-4418-8995-e541e88668c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:30.000889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7149fb9f-94f0-4418-8995-e541e88668c3 None None] MFA Rules not processed for user `7e63b8e907764f2c818391ab2ad9184c`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:30.002139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7149fb9f-94f0-4418-8995-e541e88668c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:30.080378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 661/1318] 10.222.0.47 () {58 vars in 1061 bytes} [Thu Jul 14 15:46:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:30.089401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110a748-530a-40ed-b259-864956bfc214 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:30.091009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d110a748-530a-40ed-b259-864956bfc214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:30.101000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:30.194838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110a748-530a-40ed-b259-864956bfc214 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:30.266443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7060c-9023-48d8-a22b-3d03d79c2548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:30.266686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7060c-9023-48d8-a22b-3d03d79c2548 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:30.266978 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7060c-9023-48d8-a22b-3d03d79c2548 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:30.267253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7060c-9023-48d8-a22b-3d03d79c2548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:30.268893 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-94c7060c-9023-48d8-a22b-3d03d79c2548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:30.330648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:30.377391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7060c-9023-48d8-a22b-3d03d79c2548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:30.378858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7060c-9023-48d8-a22b-3d03d79c2548 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:30.397875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 658/1319] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:33.627420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078115b-f47c-483c-a413-3dd245aaafad None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:33.627957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078115b-f47c-483c-a413-3dd245aaafad None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:33.627957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078115b-f47c-483c-a413-3dd245aaafad None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:33.629349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 662/1320] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:33.634451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415da63-f21c-430f-969f-17bd5a63b09f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:33.634880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415da63-f21c-430f-969f-17bd5a63b09f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:33.635025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415da63-f21c-430f-969f-17bd5a63b09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:33.700373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-9415da63-f21c-430f-969f-17bd5a63b09f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:33.701598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9415da63-f21c-430f-969f-17bd5a63b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:33.796424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 659/1321] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:33.835863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078115b-f47c-483c-a413-3dd245aaafad None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:33.837404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a078115b-f47c-483c-a413-3dd245aaafad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:33.887123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:33.939802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078115b-f47c-483c-a413-3dd245aaafad None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:34.011899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7d5fd-1d67-415f-8441-e5a663f61c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:34.012546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7d5fd-1d67-415f-8441-e5a663f61c27 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:34.012832 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7d5fd-1d67-415f-8441-e5a663f61c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:34.013119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7d5fd-1d67-415f-8441-e5a663f61c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:34.014793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6fb7d5fd-1d67-415f-8441-e5a663f61c27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:34.113121 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:34.126344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7d5fd-1d67-415f-8441-e5a663f61c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:34.127787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7d5fd-1d67-415f-8441-e5a663f61c27 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:34.145555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 663/1322] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:37.316688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735dbd2-7374-49a7-a8bf-6fd353d81ac5 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:37.317279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735dbd2-7374-49a7-a8bf-6fd353d81ac5 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:37.317279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735dbd2-7374-49a7-a8bf-6fd353d81ac5 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:37.318629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 660/1323] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:37.324716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df2038-8f74-4fd8-a24d-5f3b9540fffc None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:37.325172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df2038-8f74-4fd8-a24d-5f3b9540fffc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:37.325288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df2038-8f74-4fd8-a24d-5f3b9540fffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:37.389716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-06df2038-8f74-4fd8-a24d-5f3b9540fffc None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:37.390947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-06df2038-8f74-4fd8-a24d-5f3b9540fffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:37.487882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 664/1324] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:37.523753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735dbd2-7374-49a7-a8bf-6fd353d81ac5 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:37.525998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6735dbd2-7374-49a7-a8bf-6fd353d81ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:37.544952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:37.639828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735dbd2-7374-49a7-a8bf-6fd353d81ac5 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:37.720996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69d444-e72e-43b9-9b81-c5f70d1b6251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:37.721669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69d444-e72e-43b9-9b81-c5f70d1b6251 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:37.722000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69d444-e72e-43b9-9b81-c5f70d1b6251 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:37.722424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69d444-e72e-43b9-9b81-c5f70d1b6251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:37.724361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1f69d444-e72e-43b9-9b81-c5f70d1b6251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:37.802012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:37.841049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69d444-e72e-43b9-9b81-c5f70d1b6251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:37.842539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69d444-e72e-43b9-9b81-c5f70d1b6251 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:37.860783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 661/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:39.947107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb6860-bd46-43e5-89e5-aa4066b8579b None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:39.947665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb6860-bd46-43e5-89e5-aa4066b8579b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:39.947665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb6860-bd46-43e5-89e5-aa4066b8579b None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:39.949048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 665/1326] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:39.954302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95860-0896-4d0f-9e00-56eed8596fb9 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:39.954745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95860-0896-4d0f-9e00-56eed8596fb9 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:39.954993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95860-0896-4d0f-9e00-56eed8596fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:40.018783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-76f95860-0896-4d0f-9e00-56eed8596fb9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:40.020010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-76f95860-0896-4d0f-9e00-56eed8596fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:40.121571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 662/1327] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:40.165985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb6860-bd46-43e5-89e5-aa4066b8579b None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:40.167521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-61cb6860-bd46-43e5-89e5-aa4066b8579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:40.231327 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:40.275348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb6860-bd46-43e5-89e5-aa4066b8579b None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:40.349580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bced0a7-881c-477a-b13f-9c56d6a97cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:40.350260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bced0a7-881c-477a-b13f-9c56d6a97cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:40.350592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bced0a7-881c-477a-b13f-9c56d6a97cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:40.350862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bced0a7-881c-477a-b13f-9c56d6a97cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:40.352534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0bced0a7-881c-477a-b13f-9c56d6a97cc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:40.461060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:40.464502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bced0a7-881c-477a-b13f-9c56d6a97cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:40.466030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bced0a7-881c-477a-b13f-9c56d6a97cc9 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:40.487211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 666/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:42.572624 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2340c-36f4-4b29-bcce-b17817ced255 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:42.573161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2340c-36f4-4b29-bcce-b17817ced255 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:42.573161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2340c-36f4-4b29-bcce-b17817ced255 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:42.574581 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 663/1329] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:42.580181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78818f4-7156-40b6-b5b7-00be9dc232d3 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:42.580750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78818f4-7156-40b6-b5b7-00be9dc232d3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:42.580849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78818f4-7156-40b6-b5b7-00be9dc232d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:42.647926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b78818f4-7156-40b6-b5b7-00be9dc232d3 None None] MFA Rules not processed for user `83ad07150d134e288ea59a065b959f79`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:42.649129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b78818f4-7156-40b6-b5b7-00be9dc232d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:42.744073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 667/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:42.781443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2340c-36f4-4b29-bcce-b17817ced255 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:42.783067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-dae2340c-36f4-4b29-bcce-b17817ced255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:42.815785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:42.893102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2340c-36f4-4b29-bcce-b17817ced255 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:42.969987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c39611-37be-472e-8baa-850c473e74e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:42.970644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c39611-37be-472e-8baa-850c473e74e8 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:42.970930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c39611-37be-472e-8baa-850c473e74e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:42.971205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c39611-37be-472e-8baa-850c473e74e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:42.972848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c2c39611-37be-472e-8baa-850c473e74e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:43.055193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:43.087979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c39611-37be-472e-8baa-850c473e74e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:43.089446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c39611-37be-472e-8baa-850c473e74e8 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:43.109882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 664/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:45.718389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f610c-85fa-4fdc-a1fc-7f67a164981c None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:45.718908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f610c-85fa-4fdc-a1fc-7f67a164981c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:45.719098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f610c-85fa-4fdc-a1fc-7f67a164981c None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:45.720334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 668/1332] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:45.725571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf56e55-0c27-4ee5-80db-5c92ba54a166 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:45.725995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf56e55-0c27-4ee5-80db-5c92ba54a166 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:45.726161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf56e55-0c27-4ee5-80db-5c92ba54a166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:45.789600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-3cf56e55-0c27-4ee5-80db-5c92ba54a166 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:45.791327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3cf56e55-0c27-4ee5-80db-5c92ba54a166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:45.888719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 665/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:45.924619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f610c-85fa-4fdc-a1fc-7f67a164981c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:45.926227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0e6f610c-85fa-4fdc-a1fc-7f67a164981c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:45.998785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:46.034065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f610c-85fa-4fdc-a1fc-7f67a164981c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:46.115569 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef7819-7894-4fbb-a4f0-2d21f53c5108 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:46.116237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7819-7894-4fbb-a4f0-2d21f53c5108 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:46.116527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7819-7894-4fbb-a4f0-2d21f53c5108 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:46.116809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7819-7894-4fbb-a4f0-2d21f53c5108 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:46.119039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-bdef7819-7894-4fbb-a4f0-2d21f53c5108 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:46.227368 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef7819-7894-4fbb-a4f0-2d21f53c5108 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:46.228806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef7819-7894-4fbb-a4f0-2d21f53c5108 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:46.238695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:46.247354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 669/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:48.332636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fba7b-7eca-4aa3-b668-eebf9795c462 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:48.333215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fba7b-7eca-4aa3-b668-eebf9795c462 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:48.333215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fba7b-7eca-4aa3-b668-eebf9795c462 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:48.334672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 666/1335] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:48.341925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78a9ee-f42b-4bf7-b907-b0e8a7ae5cf9 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:48.341925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78a9ee-f42b-4bf7-b907-b0e8a7ae5cf9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:48.341925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78a9ee-f42b-4bf7-b907-b0e8a7ae5cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:48.403203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-af78a9ee-f42b-4bf7-b907-b0e8a7ae5cf9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:48.404397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-af78a9ee-f42b-4bf7-b907-b0e8a7ae5cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:48.499424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 670/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:48.539795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fba7b-7eca-4aa3-b668-eebf9795c462 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:48.541336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b71fba7b-7eca-4aa3-b668-eebf9795c462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:48.583633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:48.647896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fba7b-7eca-4aa3-b668-eebf9795c462 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:48.720228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6db448-dac2-4a3b-b424-8d43ae718953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:48.720875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6db448-dac2-4a3b-b424-8d43ae718953 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:48.721160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6db448-dac2-4a3b-b424-8d43ae718953 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:48.721434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6db448-dac2-4a3b-b424-8d43ae718953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:48.723102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9a6db448-dac2-4a3b-b424-8d43ae718953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:48.805118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:48.831002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6db448-dac2-4a3b-b424-8d43ae718953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:48.832464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6db448-dac2-4a3b-b424-8d43ae718953 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:48.852910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 667/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:51.611356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922fe10-eca5-4ab7-ae84-e8013d5d5274 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:51.611892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922fe10-eca5-4ab7-ae84-e8013d5d5274 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:51.611892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922fe10-eca5-4ab7-ae84-e8013d5d5274 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:51.613300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 671/1338] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:51.618540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52a4b2-e5bc-446f-808a-b6ad460e5cab None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:51.618964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52a4b2-e5bc-446f-808a-b6ad460e5cab None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:51.619082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52a4b2-e5bc-446f-808a-b6ad460e5cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:51.682739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-dc52a4b2-e5bc-446f-808a-b6ad460e5cab None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:51.683936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-dc52a4b2-e5bc-446f-808a-b6ad460e5cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:51.779286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 668/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:51.816752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922fe10-eca5-4ab7-ae84-e8013d5d5274 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:51.818260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b922fe10-eca5-4ab7-ae84-e8013d5d5274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:51.894635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:51.922176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922fe10-eca5-4ab7-ae84-e8013d5d5274 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:51.995654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96242110-fac6-4fe3-b494-3db4816466ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:51.996300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242110-fac6-4fe3-b494-3db4816466ed service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:51.996578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242110-fac6-4fe3-b494-3db4816466ed service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:51.996850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242110-fac6-4fe3-b494-3db4816466ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:51.998510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-96242110-fac6-4fe3-b494-3db4816466ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:52.108784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96242110-fac6-4fe3-b494-3db4816466ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:52.110261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96242110-fac6-4fe3-b494-3db4816466ed service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:52.128592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 672/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:56.312018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce72c44-6072-42e8-a871-07f30040fd3c None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:56.312528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce72c44-6072-42e8-a871-07f30040fd3c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:56.312528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce72c44-6072-42e8-a871-07f30040fd3c None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:56.314007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 669/1341] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:56.319320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00840b3-22eb-44fc-a0ae-99eefe0f012c None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:56.319774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00840b3-22eb-44fc-a0ae-99eefe0f012c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:56.319857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00840b3-22eb-44fc-a0ae-99eefe0f012c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:56.324657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:56.390801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c00840b3-22eb-44fc-a0ae-99eefe0f012c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:46:56.392013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c00840b3-22eb-44fc-a0ae-99eefe0f012c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:56.495647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 673/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:46:56.534872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce72c44-6072-42e8-a871-07f30040fd3c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:46:56.536414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5ce72c44-6072-42e8-a871-07f30040fd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:56.586136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:56.642333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce72c44-6072-42e8-a871-07f30040fd3c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:46:56.717788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017391e-2c12-46fa-a0c8-3c2780ba3b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:46:56.718470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017391e-2c12-46fa-a0c8-3c2780ba3b13 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:56.718757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017391e-2c12-46fa-a0c8-3c2780ba3b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:56.719036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017391e-2c12-46fa-a0c8-3c2780ba3b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:56.720687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d017391e-2c12-46fa-a0c8-3c2780ba3b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:46:56.816276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:46:56.832942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017391e-2c12-46fa-a0c8-3c2780ba3b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:46:56.834440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017391e-2c12-46fa-a0c8-3c2780ba3b13 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:46:56.854219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 670/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:46:59.953830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde632a-3f14-4bf5-bdb7-ffd944baa49f None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:59.954361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde632a-3f14-4bf5-bdb7-ffd944baa49f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:59.954473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde632a-3f14-4bf5-bdb7-ffd944baa49f None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:46:59.955797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 674/1344] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:46:59.961417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ab4ee-c288-4752-94b4-fc81c938898c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:46:59.961855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ab4ee-c288-4752-94b4-fc81c938898c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:46:59.961973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ab4ee-c288-4752-94b4-fc81c938898c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:00.026531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0f5ab4ee-c288-4752-94b4-fc81c938898c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:00.027665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0f5ab4ee-c288-4752-94b4-fc81c938898c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:00.130848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 671/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:46:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:00.168724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde632a-3f14-4bf5-bdb7-ffd944baa49f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:00.170291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4bde632a-3f14-4bf5-bdb7-ffd944baa49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:00.258836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:00.276455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde632a-3f14-4bf5-bdb7-ffd944baa49f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:00.353155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7568e-f62f-45b5-93e8-89eb5737290a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:00.353837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7568e-f62f-45b5-93e8-89eb5737290a service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:00.354125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7568e-f62f-45b5-93e8-89eb5737290a service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:00.354402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7568e-f62f-45b5-93e8-89eb5737290a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:00.356130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-78b7568e-f62f-45b5-93e8-89eb5737290a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:00.472056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7568e-f62f-45b5-93e8-89eb5737290a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:00.473514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7568e-f62f-45b5-93e8-89eb5737290a service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:00.492219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 675/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:05.204444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bda6b-8c2c-42d7-88a2-5b325d288329 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:05.204980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bda6b-8c2c-42d7-88a2-5b325d288329 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:05.204980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35bda6b-8c2c-42d7-88a2-5b325d288329 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:05.206445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 672/1347] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:05.212860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eda691-b214-473e-9db5-34e4e1e858d1 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:05.213316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eda691-b214-473e-9db5-34e4e1e858d1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:05.213474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eda691-b214-473e-9db5-34e4e1e858d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:05.230422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:05.303289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-60eda691-b214-473e-9db5-34e4e1e858d1 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:05.304561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-60eda691-b214-473e-9db5-34e4e1e858d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:05.422853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 676/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:05.464776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bda6b-8c2c-42d7-88a2-5b325d288329 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:05.466460 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a35bda6b-8c2c-42d7-88a2-5b325d288329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:05.529464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:05.583600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35bda6b-8c2c-42d7-88a2-5b325d288329 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:05.665311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ece54-20a0-4c96-a592-036cf3cfedc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:05.666425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ece54-20a0-4c96-a592-036cf3cfedc7 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:05.666919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ece54-20a0-4c96-a592-036cf3cfedc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:05.667404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ece54-20a0-4c96-a592-036cf3cfedc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:05.670518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-869ece54-20a0-4c96-a592-036cf3cfedc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:05.786026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:05.790201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ece54-20a0-4c96-a592-036cf3cfedc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:05.791658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ece54-20a0-4c96-a592-036cf3cfedc7 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:05.809985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 673/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:10.661792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4e3c4-cbbf-4d5d-9dd2-766781dfe96a None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:10.662643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4e3c4-cbbf-4d5d-9dd2-766781dfe96a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:10.662930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4e3c4-cbbf-4d5d-9dd2-766781dfe96a None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:10.665034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 677/1350] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:47:10.673096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8bf52-91c3-4b8a-8b20-9b8c5be370f1 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:10.673096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8bf52-91c3-4b8a-8b20-9b8c5be370f1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:10.673096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8bf52-91c3-4b8a-8b20-9b8c5be370f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:10.761569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-fcd8bf52-91c3-4b8a-8b20-9b8c5be370f1 None None] MFA Rules not processed for user `97524081e59f4775b83d82edf737810e`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:10.763093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fcd8bf52-91c3-4b8a-8b20-9b8c5be370f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:10.913905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 674/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:47:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:15.198464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97781c83-b299-49cf-95d2-fc761fb6b0a7 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:15.199406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97781c83-b299-49cf-95d2-fc761fb6b0a7 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:15.199833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97781c83-b299-49cf-95d2-fc761fb6b0a7 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:15.201292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 678/1352] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:15.206838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a484318-79fd-4af9-a58f-a796be9d6b26 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:15.207352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a484318-79fd-4af9-a58f-a796be9d6b26 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:15.207438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a484318-79fd-4af9-a58f-a796be9d6b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:15.267639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:15.297056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-1a484318-79fd-4af9-a58f-a796be9d6b26 None None] MFA Rules not processed for user `97524081e59f4775b83d82edf737810e`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:15.297848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1a484318-79fd-4af9-a58f-a796be9d6b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:15.415280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 675/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:47:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:15.420653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=59 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:16.566782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d0e30-569a-4f43-8170-9b15f5a136bf None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:16.567449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d0e30-569a-4f43-8170-9b15f5a136bf None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:16.567449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d0e30-569a-4f43-8170-9b15f5a136bf None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:16.569054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 679/1354] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:16.576297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d12ed2-2728-4430-9537-e643b4669793 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:16.576818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d12ed2-2728-4430-9537-e643b4669793 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:16.577322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d12ed2-2728-4430-9537-e643b4669793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:16.657044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-34d12ed2-2728-4430-9537-e643b4669793 None None] MFA Rules not processed for user `97524081e59f4775b83d82edf737810e`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:16.658262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-34d12ed2-2728-4430-9537-e643b4669793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:16.701953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:16.767939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 676/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:47:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:20.184424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca26d1-3278-431f-bb89-f55dd30bd367 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:20.185359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca26d1-3278-431f-bb89-f55dd30bd367 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:20.185918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca26d1-3278-431f-bb89-f55dd30bd367 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:20.187802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 680/1356] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:20.193496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7febc7-119f-45b4-853d-ea0b66e6d612 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:20.194006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7febc7-119f-45b4-853d-ea0b66e6d612 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:20.194571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7febc7-119f-45b4-853d-ea0b66e6d612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:20.269057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ad7febc7-119f-45b4-853d-ea0b66e6d612 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:20.274252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ad7febc7-119f-45b4-853d-ea0b66e6d612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:20.385847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 677/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:20.435208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41e86-570b-47bd-8ec8-94f6f98395cf None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:20.435897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41e86-570b-47bd-8ec8-94f6f98395cf None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:20.436395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41e86-570b-47bd-8ec8-94f6f98395cf None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:20.438954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 681/1358] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jul 14 15:47:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:20.448303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c81007-32bc-4409-b37f-ca2ed738fbfc None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:20.449295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c81007-32bc-4409-b37f-ca2ed738fbfc None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:20.450515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c81007-32bc-4409-b37f-ca2ed738fbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:20.468061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:20.534536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f4c81007-32bc-4409-b37f-ca2ed738fbfc None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:20.535830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f4c81007-32bc-4409-b37f-ca2ed738fbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:20.670523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 678/1359] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jul 14 15:47:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:20.678592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b41e86-570b-47bd-8ec8-94f6f98395cf None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:20.680183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b6b41e86-570b-47bd-8ec8-94f6f98395cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:20.836654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b41e86-570b-47bd-8ec8-94f6f98395cf None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:20.911923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:20.953554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad283405-fbe1-44e5-8f00-f90d2560280c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:20.954257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad283405-fbe1-44e5-8f00-f90d2560280c service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:20.954545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad283405-fbe1-44e5-8f00-f90d2560280c service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:20.954870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad283405-fbe1-44e5-8f00-f90d2560280c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:20.956769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ad283405-fbe1-44e5-8f00-f90d2560280c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:21.060826 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad283405-fbe1-44e5-8f00-f90d2560280c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:21.062613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad283405-fbe1-44e5-8f00-f90d2560280c service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:21.090015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 682/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:23.342771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b5836-0330-4916-9089-34d5e360284c None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:23.343619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b5836-0330-4916-9089-34d5e360284c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:23.344076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b5836-0330-4916-9089-34d5e360284c None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:23.345720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 679/1361] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:23.351287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab12f3-d935-4acf-bc92-1916687ede03 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:23.352494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab12f3-d935-4acf-bc92-1916687ede03 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:23.352494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab12f3-d935-4acf-bc92-1916687ede03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:23.420854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a5ab12f3-d935-4acf-bc92-1916687ede03 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:23.422097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a5ab12f3-d935-4acf-bc92-1916687ede03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:23.463586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:23.532461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 683/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:23.578757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b5836-0330-4916-9089-34d5e360284c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:23.580341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-401b5836-0330-4916-9089-34d5e360284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:23.654291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:23.749258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b5836-0330-4916-9089-34d5e360284c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:23.899239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b2297-66cf-48b7-8587-50931d7adf44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:23.900052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b2297-66cf-48b7-8587-50931d7adf44 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:23.900444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b2297-66cf-48b7-8587-50931d7adf44 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:23.900826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b2297-66cf-48b7-8587-50931d7adf44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:23.902637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c58b2297-66cf-48b7-8587-50931d7adf44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:23.926488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:24.001399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b2297-66cf-48b7-8587-50931d7adf44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:24.003427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b2297-66cf-48b7-8587-50931d7adf44 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:24.024970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 680/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:26.531705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91854c0c-c464-40f3-b03d-d9b676ed38ac None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:26.532239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91854c0c-c464-40f3-b03d-d9b676ed38ac None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:26.532360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91854c0c-c464-40f3-b03d-d9b676ed38ac None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:26.533971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 684/1364] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:26.542098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1a44e-f5bf-4f2d-a131-6de85434c497 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:26.542567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1a44e-f5bf-4f2d-a131-6de85434c497 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:26.542822 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1a44e-f5bf-4f2d-a131-6de85434c497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:26.618718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-b7d1a44e-f5bf-4f2d-a131-6de85434c497 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:26.620088 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b7d1a44e-f5bf-4f2d-a131-6de85434c497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:26.722902 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:26.725856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 681/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:26.775260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f26846-a2eb-46c2-9c13-d670ee658deb None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:26.775776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f26846-a2eb-46c2-9c13-d670ee658deb None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:26.776190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f26846-a2eb-46c2-9c13-d670ee658deb None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:26.777857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 685/1366] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jul 14 15:47:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:26.783383 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59796f38-4e47-40ab-ad36-d340763b325d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:26.783618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59796f38-4e47-40ab-ad36-d340763b325d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:26.783898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59796f38-4e47-40ab-ad36-d340763b325d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:26.865058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-59796f38-4e47-40ab-ad36-d340763b325d None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:26.866319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-59796f38-4e47-40ab-ad36-d340763b325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:26.995010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 682/1367] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jul 14 15:47:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:27.002973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f26846-a2eb-46c2-9c13-d670ee658deb None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:27.004922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-19f26846-a2eb-46c2-9c13-d670ee658deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:27.171256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f26846-a2eb-46c2-9c13-d670ee658deb None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:27.193935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:27.296576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b9111-b4c0-43a7-a7fb-721048b13ab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:27.297621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b9111-b4c0-43a7-a7fb-721048b13ab8 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:27.298016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b9111-b4c0-43a7-a7fb-721048b13ab8 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:27.298363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b9111-b4c0-43a7-a7fb-721048b13ab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:27.300096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d66b9111-b4c0-43a7-a7fb-721048b13ab8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:27.398956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b9111-b4c0-43a7-a7fb-721048b13ab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:27.400562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b9111-b4c0-43a7-a7fb-721048b13ab8 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:27.421697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 686/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:47:28.342438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b25cd-620e-4beb-b548-2994d2378995 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:28.342980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b25cd-620e-4beb-b548-2994d2378995 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:28.342980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b25cd-620e-4beb-b548-2994d2378995 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:28.344453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 683/1369] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:28.349648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d6406-f24c-4e1d-93bc-ab243d3069e1 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:28.350124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d6406-f24c-4e1d-93bc-ab243d3069e1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:28.350223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d6406-f24c-4e1d-93bc-ab243d3069e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:28.382454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:28.418572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-303d6406-f24c-4e1d-93bc-ab243d3069e1 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:28.420178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-303d6406-f24c-4e1d-93bc-ab243d3069e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:28.523221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 687/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:28.877278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3548cdc-a3b8-4860-9554-651a5c4fcb70 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:28.877278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3548cdc-a3b8-4860-9554-651a5c4fcb70 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:28.877628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3548cdc-a3b8-4860-9554-651a5c4fcb70 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:28.879192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 684/1371] 10.222.0.47 () {54 vars in 963 bytes} [Thu Jul 14 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:28.884650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1431fad-6f6d-4147-8bb2-7c6e609390a1 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:28.885049 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1431fad-6f6d-4147-8bb2-7c6e609390a1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:28.885445 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1431fad-6f6d-4147-8bb2-7c6e609390a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:28.965285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d1431fad-6f6d-4147-8bb2-7c6e609390a1 None None] MFA Rules not processed for user `3d54d684efce44b99f3235e80beb1c56`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:28.966700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d1431fad-6f6d-4147-8bb2-7c6e609390a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:29.015655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:29.050592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 688/1372] 10.222.0.47 () {58 vars in 1060 bytes} [Thu Jul 14 15:47:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:47:29.059649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3548cdc-a3b8-4860-9554-651a5c4fcb70 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:29.061255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f3548cdc-a3b8-4860-9554-651a5c4fcb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:29.115155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:29.173313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3548cdc-a3b8-4860-9554-651a5c4fcb70 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:29.252570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8532d-3e7c-4f54-9093-2beabc257da9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:29.253392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8532d-3e7c-4f54-9093-2beabc257da9 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:29.253798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8532d-3e7c-4f54-9093-2beabc257da9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:29.254196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8532d-3e7c-4f54-9093-2beabc257da9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:29.255965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d0f8532d-3e7c-4f54-9093-2beabc257da9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:29.363686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:29.379622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8532d-3e7c-4f54-9093-2beabc257da9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:29.381205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8532d-3e7c-4f54-9093-2beabc257da9 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:29.402592 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 685/1373] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:29.766357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8263ad-8f0a-4f7a-bb68-58b7f4e19ebc None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:29.766357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8263ad-8f0a-4f7a-bb68-58b7f4e19ebc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:29.766357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8263ad-8f0a-4f7a-bb68-58b7f4e19ebc None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:29.767440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 689/1374] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:29.772247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93a16-0112-469f-8fa8-d58ccee84fcf None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:29.772533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93a16-0112-469f-8fa8-d58ccee84fcf None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:29.772983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93a16-0112-469f-8fa8-d58ccee84fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:29.846326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ebd93a16-0112-469f-8fa8-d58ccee84fcf None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:29.847559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ebd93a16-0112-469f-8fa8-d58ccee84fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:29.944638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 686/1375] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:29.988106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8263ad-8f0a-4f7a-bb68-58b7f4e19ebc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:29.989618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-de8263ad-8f0a-4f7a-bb68-58b7f4e19ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:30.139787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8263ad-8f0a-4f7a-bb68-58b7f4e19ebc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:30.192301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:30.253986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cd723-b3f6-40ad-ab4d-cbbc298b29e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:30.254705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cd723-b3f6-40ad-ab4d-cbbc298b29e0 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:30.254987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cd723-b3f6-40ad-ab4d-cbbc298b29e0 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:30.255262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cd723-b3f6-40ad-ab4d-cbbc298b29e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:30.256967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-bb4cd723-b3f6-40ad-ab4d-cbbc298b29e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:30.348680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cd723-b3f6-40ad-ab4d-cbbc298b29e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:30.350408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cd723-b3f6-40ad-ab4d-cbbc298b29e0 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:30.368853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 690/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:32.584906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35287-07d6-48e2-ba6c-546fafe691ea None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:32.585417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35287-07d6-48e2-ba6c-546fafe691ea None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:32.585417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35287-07d6-48e2-ba6c-546fafe691ea None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:32.586842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 687/1377] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:32.592045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f58fc-d9ca-48e3-9105-6a223b02e06e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:32.592463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f58fc-d9ca-48e3-9105-6a223b02e06e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:32.592634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f58fc-d9ca-48e3-9105-6a223b02e06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:32.657323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-422f58fc-d9ca-48e3-9105-6a223b02e06e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:32.658553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-422f58fc-d9ca-48e3-9105-6a223b02e06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:32.665245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:32.755753 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 691/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:32.802129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc35287-07d6-48e2-ba6c-546fafe691ea None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:32.803798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-dbc35287-07d6-48e2-ba6c-546fafe691ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:32.865110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:32.990228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc35287-07d6-48e2-ba6c-546fafe691ea None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:33.131824 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211b09c-164e-45aa-a554-c246a266c864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:33.132401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211b09c-164e-45aa-a554-c246a266c864 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:33.133206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211b09c-164e-45aa-a554-c246a266c864 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:33.133487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211b09c-164e-45aa-a554-c246a266c864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:33.135188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1211b09c-164e-45aa-a554-c246a266c864 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:33.155547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:33.231559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211b09c-164e-45aa-a554-c246a266c864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:33.233185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211b09c-164e-45aa-a554-c246a266c864 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:33.255072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 688/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:35.523937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972828ba-0689-4612-abda-65fdbe0556b3 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:35.524385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972828ba-0689-4612-abda-65fdbe0556b3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:35.524506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972828ba-0689-4612-abda-65fdbe0556b3 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:35.525957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 692/1380] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:35.531225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f72a7-76fb-483f-884d-5040ee189b8d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:35.531640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f72a7-76fb-483f-884d-5040ee189b8d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:35.531791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f72a7-76fb-483f-884d-5040ee189b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:35.604330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d49f72a7-76fb-483f-884d-5040ee189b8d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:35.605543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d49f72a7-76fb-483f-884d-5040ee189b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:35.707541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:35.714221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 689/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:35.759655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972828ba-0689-4612-abda-65fdbe0556b3 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:35.761357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-972828ba-0689-4612-abda-65fdbe0556b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:35.907587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:35.918963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972828ba-0689-4612-abda-65fdbe0556b3 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:36.061538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fe698-a0d9-471e-8526-6f9b7f73317d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:36.062257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe698-a0d9-471e-8526-6f9b7f73317d service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:36.062538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe698-a0d9-471e-8526-6f9b7f73317d service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:36.062944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fe698-a0d9-471e-8526-6f9b7f73317d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:36.064654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2e6fe698-a0d9-471e-8526-6f9b7f73317d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:36.162245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fe698-a0d9-471e-8526-6f9b7f73317d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:36.163755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fe698-a0d9-471e-8526-6f9b7f73317d service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:36.174988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:36.184785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 693/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:38.462777 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4cdb2-ca7c-4064-b0a1-91994a90e2c6 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:38.463507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4cdb2-ca7c-4064-b0a1-91994a90e2c6 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:38.463507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4cdb2-ca7c-4064-b0a1-91994a90e2c6 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:38.464742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 690/1383] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:38.470107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be4c3-7a13-4624-9d92-31e46d4f669f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:38.470624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be4c3-7a13-4624-9d92-31e46d4f669f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:38.470790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be4c3-7a13-4624-9d92-31e46d4f669f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:38.536069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-652be4c3-7a13-4624-9d92-31e46d4f669f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:38.537319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-652be4c3-7a13-4624-9d92-31e46d4f669f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:38.642749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 694/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:38.685356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4cdb2-ca7c-4064-b0a1-91994a90e2c6 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:38.686915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-50b4cdb2-ca7c-4064-b0a1-91994a90e2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:38.854312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4cdb2-ca7c-4064-b0a1-91994a90e2c6 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:38.939123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:38.996061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc22abd-cb4c-4295-abd8-23a721342106 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:38.996927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc22abd-cb4c-4295-abd8-23a721342106 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:38.997309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc22abd-cb4c-4295-abd8-23a721342106 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:38.997663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc22abd-cb4c-4295-abd8-23a721342106 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:38.999923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-cdc22abd-cb4c-4295-abd8-23a721342106 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:39.105965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc22abd-cb4c-4295-abd8-23a721342106 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:39.107471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc22abd-cb4c-4295-abd8-23a721342106 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:39.126112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 691/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:41.340647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b32f8d-d196-46ff-a97d-c227e98b1a0f None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:41.341202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b32f8d-d196-46ff-a97d-c227e98b1a0f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:41.341306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b32f8d-d196-46ff-a97d-c227e98b1a0f None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:41.342828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 695/1386] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:41.348221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28c5c-ffd3-49dc-a4dd-2dd201755260 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:41.348643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28c5c-ffd3-49dc-a4dd-2dd201755260 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:41.348786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28c5c-ffd3-49dc-a4dd-2dd201755260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:41.415452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-36d28c5c-ffd3-49dc-a4dd-2dd201755260 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:41.416674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-36d28c5c-ffd3-49dc-a4dd-2dd201755260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:41.450682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:41.522476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 692/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:41.569291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b32f8d-d196-46ff-a97d-c227e98b1a0f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:41.570882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b5b32f8d-d196-46ff-a97d-c227e98b1a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:41.652888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:41.728867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b32f8d-d196-46ff-a97d-c227e98b1a0f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:41.854567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83715a-d6ee-4bf0-9df6-18b942db217b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:41.855145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83715a-d6ee-4bf0-9df6-18b942db217b service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:41.855425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83715a-d6ee-4bf0-9df6-18b942db217b service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:41.855702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83715a-d6ee-4bf0-9df6-18b942db217b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:41.857362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fa83715a-d6ee-4bf0-9df6-18b942db217b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:41.896756 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:41.950153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83715a-d6ee-4bf0-9df6-18b942db217b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:41.951774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83715a-d6ee-4bf0-9df6-18b942db217b service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:41.970630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 696/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:44.156718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1220bd-af22-483c-9e1e-063845390f78 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:44.157252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1220bd-af22-483c-9e1e-063845390f78 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:44.157366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1220bd-af22-483c-9e1e-063845390f78 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:44.158734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 693/1389] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:44.164213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f588-a299-43c8-9a7b-50b707e2bc77 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:44.164651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f588-a299-43c8-9a7b-50b707e2bc77 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:44.164766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f588-a299-43c8-9a7b-50b707e2bc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:44.230907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-db84f588-a299-43c8-9a7b-50b707e2bc77 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:44.232138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-db84f588-a299-43c8-9a7b-50b707e2bc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:44.332085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 697/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:44.376397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1220bd-af22-483c-9e1e-063845390f78 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:44.378271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9a1220bd-af22-483c-9e1e-063845390f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:44.526654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1220bd-af22-483c-9e1e-063845390f78 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:44.542946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:44.643389 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63bdd2-43a0-42a9-88f3-3e18c63fce70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:44.643980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63bdd2-43a0-42a9-88f3-3e18c63fce70 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:44.644268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63bdd2-43a0-42a9-88f3-3e18c63fce70 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:44.644541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63bdd2-43a0-42a9-88f3-3e18c63fce70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:44.646228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5a63bdd2-43a0-42a9-88f3-3e18c63fce70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:44.740730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63bdd2-43a0-42a9-88f3-3e18c63fce70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:44.741678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a83291-decd-4315-b92a-774b25d6f1ca None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:44.742500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a83291-decd-4315-b92a-774b25d6f1ca None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:44.742778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a83291-decd-4315-b92a-774b25d6f1ca None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:44.742928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63bdd2-43a0-42a9-88f3-3e18c63fce70 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:44.744155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 698/1391] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:47:44.749231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a782b-ae36-449e-9c3c-15303866fd66 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:44.749526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a782b-ae36-449e-9c3c-15303866fd66 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:44.749806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a782b-ae36-449e-9c3c-15303866fd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:44.761656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 694/1392] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:44.763871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:44.815272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-003a782b-ae36-449e-9c3c-15303866fd66 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:44.816521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-003a782b-ae36-449e-9c3c-15303866fd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:44.914471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 699/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:47:44.973696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff403fb4-d21d-46ad-b1ff-567ba451cdb2 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:44.977275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff403fb4-d21d-46ad-b1ff-567ba451cdb2 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:44.977275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff403fb4-d21d-46ad-b1ff-567ba451cdb2 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:44.977275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 695/1394] 10.222.0.47 () {54 vars in 962 bytes} [Thu Jul 14 15:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:44.981126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc93fc-2ef1-419c-a143-9157e9d54e8b None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:44.981417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc93fc-2ef1-419c-a143-9157e9d54e8b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:44.981766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc93fc-2ef1-419c-a143-9157e9d54e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:45.068564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d8cc93fc-2ef1-419c-a143-9157e9d54e8b None None] MFA Rules not processed for user `5eeda3264191409eba35ffbb908770a6`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:45.070428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d8cc93fc-2ef1-419c-a143-9157e9d54e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:45.111591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:45.156342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 700/1395] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Jul 14 15:47:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:45.163899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff403fb4-d21d-46ad-b1ff-567ba451cdb2 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:45.165507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ff403fb4-d21d-46ad-b1ff-567ba451cdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:45.191580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:45.282785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff403fb4-d21d-46ad-b1ff-567ba451cdb2 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:45.361593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7397cc-50d3-48b2-bbe0-d4b62e5bd2b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5eeda3264191409eba35ffbb908770a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:45.362290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7397cc-50d3-48b2-bbe0-d4b62e5bd2b7 service glance] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:45.362580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7397cc-50d3-48b2-bbe0-d4b62e5bd2b7 service glance] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:45.362859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7397cc-50d3-48b2-bbe0-d4b62e5bd2b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:45.364526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fe7397cc-50d3-48b2-bbe0-d4b62e5bd2b7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:45.440985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:45.477898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7397cc-50d3-48b2-bbe0-d4b62e5bd2b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:45.479361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7397cc-50d3-48b2-bbe0-d4b62e5bd2b7 service glance] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:45.497863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 696/1396] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:46.372139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b8108-b348-4679-88a6-44fe293e2cf7 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:46.372628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b8108-b348-4679-88a6-44fe293e2cf7 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:46.372835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b8108-b348-4679-88a6-44fe293e2cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:46.439451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-1d0b8108-b348-4679-88a6-44fe293e2cf7 None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:46.440668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1d0b8108-b348-4679-88a6-44fe293e2cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:46.528435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 701/1397] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:47:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:46.580621 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a880a-ea31-4ae8-b6d7-b0ba2e300636 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:46.581090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a880a-ea31-4ae8-b6d7-b0ba2e300636 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:46.581187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a880a-ea31-4ae8-b6d7-b0ba2e300636 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:46.582773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 697/1398] 10.222.0.47 () {54 vars in 942 bytes} [Thu Jul 14 15:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:46.589764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c08675-2229-4597-a1fb-155f6e4b6e51 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:46.590118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c08675-2229-4597-a1fb-155f6e4b6e51 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:46.590401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c08675-2229-4597-a1fb-155f6e4b6e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:46.651284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:46.670658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-82c08675-2229-4597-a1fb-155f6e4b6e51 None None] MFA Rules not processed for user `95f36b4517ee4b64bc43e000c854df30`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:46.672025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-82c08675-2229-4597-a1fb-155f6e4b6e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:46.780686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 702/1399] 10.222.0.47 () {58 vars in 1039 bytes} [Thu Jul 14 15:47:46 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:46.790317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a880a-ea31-4ae8-b6d7-b0ba2e300636 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:46.791835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-829a880a-ea31-4ae8-b6d7-b0ba2e300636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:46.928249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a880a-ea31-4ae8-b6d7-b0ba2e300636 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:46.971462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:47.006574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cafc03-a676-46ad-829e-2fa1b8a34921 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:47.006867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cafc03-a676-46ad-829e-2fa1b8a34921 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:47.007146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cafc03-a676-46ad-829e-2fa1b8a34921 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:47.009110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 703/1400] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:47:47.013986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecbfd5-8ac6-4f2b-bb0c-d9807fb045a0 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:47.014282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecbfd5-8ac6-4f2b-bb0c-d9807fb045a0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:47.014568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecbfd5-8ac6-4f2b-bb0c-d9807fb045a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:47.039510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:47.040085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:47.040368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:47.040652 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:47.042410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:47.084694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e7ecbfd5-8ac6-4f2b-bb0c-d9807fb045a0 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:47.085960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e7ecbfd5-8ac6-4f2b-bb0c-d9807fb045a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:47.127657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:47.129633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:47.138401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 698/1401] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:47:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:47:47.144941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:47.188160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 704/1402] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:47.232611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:47.233901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:47.305197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:47.327505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7c4b8-5ba4-4d24-adff-5136af33fc33 service swift] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:47.441747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7badc60-19cf-4c63-9049-51e708b43831 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:47.442441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7badc60-19cf-4c63-9049-51e708b43831 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:47.442730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7badc60-19cf-4c63-9049-51e708b43831 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:47.443019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7badc60-19cf-4c63-9049-51e708b43831 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:47.444684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d7badc60-19cf-4c63-9049-51e708b43831 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:47.535610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7badc60-19cf-4c63-9049-51e708b43831 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:47.537240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7badc60-19cf-4c63-9049-51e708b43831 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:47.543693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:47.557594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 699/1403] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:49.781729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d6b45-5840-472a-8696-645d13a58939 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:49.782367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d6b45-5840-472a-8696-645d13a58939 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:49.782367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d6b45-5840-472a-8696-645d13a58939 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:49.783711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 705/1404] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:49.789844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ab1c9-d0c8-4a77-809c-c742cb9c24f6 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:49.790293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ab1c9-d0c8-4a77-809c-c742cb9c24f6 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:49.790473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ab1c9-d0c8-4a77-809c-c742cb9c24f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:49.868159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-b48ab1c9-d0c8-4a77-809c-c742cb9c24f6 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:49.869733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b48ab1c9-d0c8-4a77-809c-c742cb9c24f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:49.984771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 700/1405] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:50.029953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d6b45-5840-472a-8696-645d13a58939 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:50.031905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e71d6b45-5840-472a-8696-645d13a58939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:50.198057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d6b45-5840-472a-8696-645d13a58939 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:50.246563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:50.328249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c62dc1-b066-41d9-9515-14d89418f45c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:50.328958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c62dc1-b066-41d9-9515-14d89418f45c service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:50.329258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c62dc1-b066-41d9-9515-14d89418f45c service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:50.329546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c62dc1-b066-41d9-9515-14d89418f45c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:50.331318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f5c62dc1-b066-41d9-9515-14d89418f45c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:50.437892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c62dc1-b066-41d9-9515-14d89418f45c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:50.439709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c62dc1-b066-41d9-9515-14d89418f45c service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:50.462985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 706/1406] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:47:52.705141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430d935-f3e4-4837-959b-0ec8a2f0d15d None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:52.705685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430d935-f3e4-4837-959b-0ec8a2f0d15d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:52.705685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430d935-f3e4-4837-959b-0ec8a2f0d15d None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:52.707121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 701/1407] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:47:52.712923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7415ea-856a-430a-ad26-69abb78b6d60 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:52.713369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7415ea-856a-430a-ad26-69abb78b6d60 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:52.713612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7415ea-856a-430a-ad26-69abb78b6d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:52.774906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:52.787390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-eb7415ea-856a-430a-ad26-69abb78b6d60 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:47:52.789194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-eb7415ea-856a-430a-ad26-69abb78b6d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:52.897627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 707/1408] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:47:52.944359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430d935-f3e4-4837-959b-0ec8a2f0d15d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:47:52.945960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0430d935-f3e4-4837-959b-0ec8a2f0d15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:53.026051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:53.114096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430d935-f3e4-4837-959b-0ec8a2f0d15d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:47:53.245425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c898b91-d1c4-44b2-90a5-6e73c09f4a05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:47:53.246121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c898b91-d1c4-44b2-90a5-6e73c09f4a05 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:47:53.246410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c898b91-d1c4-44b2-90a5-6e73c09f4a05 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:47:53.246699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c898b91-d1c4-44b2-90a5-6e73c09f4a05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:47:53.248353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3c898b91-d1c4-44b2-90a5-6e73c09f4a05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:47:53.286742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:47:53.353123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c898b91-d1c4-44b2-90a5-6e73c09f4a05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:47:53.354771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c898b91-d1c4-44b2-90a5-6e73c09f4a05 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:47:53.374368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 702/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:02.895809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=44 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:03.371539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=31 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:10.958168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958bff6-e43c-446f-b805-72170fa9526b None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:10.958744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958bff6-e43c-446f-b805-72170fa9526b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:10.958744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958bff6-e43c-446f-b805-72170fa9526b None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:10.960146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 708/1410] 10.222.0.47 () {54 vars in 918 bytes} [Thu Jul 14 15:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:10.965206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0274be7-4a48-407f-927a-0731778b9980 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:10.965643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0274be7-4a48-407f-927a-0731778b9980 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:10.965750 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0274be7-4a48-407f-927a-0731778b9980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:11.043303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d0274be7-4a48-407f-927a-0731778b9980 None None] MFA Rules not processed for user `97524081e59f4775b83d82edf737810e`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:11.044779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d0274be7-4a48-407f-927a-0731778b9980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:11.154661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 703/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jul 14 15:48:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:11.169638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd6bc6-2b56-4a61-9e61-2ac6741e5d29 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:11.169959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd6bc6-2b56-4a61-9e61-2ac6741e5d29 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:11.170241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd6bc6-2b56-4a61-9e61-2ac6741e5d29 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:11.171612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 709/1412] 10.222.0.47 () {54 vars in 918 bytes} [Thu Jul 14 15:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:11.176221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70767f-490d-46a4-a841-d7828fec385f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:11.176546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70767f-490d-46a4-a841-d7828fec385f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:11.176830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70767f-490d-46a4-a841-d7828fec385f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:11.248959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:11.251773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ba70767f-490d-46a4-a841-d7828fec385f None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:11.253028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ba70767f-490d-46a4-a841-d7828fec385f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:11.369750 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 704/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jul 14 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:13.844933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87289e6e-ea82-4596-aef3-d556d9eec0c2 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:13.845495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87289e6e-ea82-4596-aef3-d556d9eec0c2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:13.845495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87289e6e-ea82-4596-aef3-d556d9eec0c2 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:13.846922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:13.853012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a48e5-c774-485e-a937-5174dd02fd9b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:13.853445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a48e5-c774-485e-a937-5174dd02fd9b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:13.853584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a48e5-c774-485e-a937-5174dd02fd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:13.920010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-417a48e5-c774-485e-a937-5174dd02fd9b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:13.921260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-417a48e5-c774-485e-a937-5174dd02fd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:13.976567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:14.018329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:14.062400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87289e6e-ea82-4596-aef3-d556d9eec0c2 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:14.063965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-87289e6e-ea82-4596-aef3-d556d9eec0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:14.208431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87289e6e-ea82-4596-aef3-d556d9eec0c2 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:14.279797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:14.319950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6c5d1-273a-4033-8ba4-8c1f91a33391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:14.320600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6c5d1-273a-4033-8ba4-8c1f91a33391 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:14.320881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6c5d1-273a-4033-8ba4-8c1f91a33391 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:14.321154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6c5d1-273a-4033-8ba4-8c1f91a33391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:14.322812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8dc6c5d1-273a-4033-8ba4-8c1f91a33391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:14.411841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6c5d1-273a-4033-8ba4-8c1f91a33391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:14.413977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6c5d1-273a-4033-8ba4-8c1f91a33391 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:14.432410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 711/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:15.329256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51504b-a82d-4a90-94f3-529d7c93ea00 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:15.329760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51504b-a82d-4a90-94f3-529d7c93ea00 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:15.329760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51504b-a82d-4a90-94f3-529d7c93ea00 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:15.331169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:15.336066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7a8df-3d31-44d2-a621-eed29add997f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:15.336495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7a8df-3d31-44d2-a621-eed29add997f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:15.336610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7a8df-3d31-44d2-a621-eed29add997f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:15.409709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-94b7a8df-3d31-44d2-a621-eed29add997f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:15.410944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-94b7a8df-3d31-44d2-a621-eed29add997f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:15.437946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:15.508281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:15.658105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51504b-a82d-4a90-94f3-529d7c93ea00 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:15.659580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2a51504b-a82d-4a90-94f3-529d7c93ea00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:15.804622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51504b-a82d-4a90-94f3-529d7c93ea00 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:15.844616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:15.917620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac808c-5179-4a0c-ae79-ffa580152e26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:15.918237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac808c-5179-4a0c-ae79-ffa580152e26 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:15.918523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac808c-5179-4a0c-ae79-ffa580152e26 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:15.918800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac808c-5179-4a0c-ae79-ffa580152e26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:15.920460 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-baac808c-5179-4a0c-ae79-ffa580152e26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:16.010282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac808c-5179-4a0c-ae79-ffa580152e26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:16.011882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac808c-5179-4a0c-ae79-ffa580152e26 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:16.030296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 707/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:16.915282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd8b26-4e37-4c21-9c61-d099a2a25879 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:16.915857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd8b26-4e37-4c21-9c61-d099a2a25879 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:16.915857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd8b26-4e37-4c21-9c61-d099a2a25879 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:16.917235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:16.922117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dded1c2-582d-47c8-9726-e6566bfc605c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:16.922537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dded1c2-582d-47c8-9726-e6566bfc605c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:16.922684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dded1c2-582d-47c8-9726-e6566bfc605c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:16.975785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:16.998265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-4dded1c2-582d-47c8-9726-e6566bfc605c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:16.999487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4dded1c2-582d-47c8-9726-e6566bfc605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:17.097862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:17.252914 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd8b26-4e37-4c21-9c61-d099a2a25879 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:17.254337 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cebd8b26-4e37-4c21-9c61-d099a2a25879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:17.399348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd8b26-4e37-4c21-9c61-d099a2a25879 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:17.411238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:17.510986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00499f8c-c369-44a5-b259-eab92a59db49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:17.511566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00499f8c-c369-44a5-b259-eab92a59db49 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:17.511849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00499f8c-c369-44a5-b259-eab92a59db49 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:17.512124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00499f8c-c369-44a5-b259-eab92a59db49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:17.513778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-00499f8c-c369-44a5-b259-eab92a59db49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:17.602795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00499f8c-c369-44a5-b259-eab92a59db49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:17.604388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00499f8c-c369-44a5-b259-eab92a59db49 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:17.622219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 714/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:18.472895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1eab-dc39-4fd2-97cb-d51d14bf7939 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:18.473425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1eab-dc39-4fd2-97cb-d51d14bf7939 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:18.473545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a1eab-dc39-4fd2-97cb-d51d14bf7939 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:18.474894 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:18.479902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010f61f-310d-4383-a3bd-d8dff3d3a255 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:18.480339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010f61f-310d-4383-a3bd-d8dff3d3a255 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:18.480477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010f61f-310d-4383-a3bd-d8dff3d3a255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:18.499302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:18.554621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0010f61f-310d-4383-a3bd-d8dff3d3a255 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:18.555841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0010f61f-310d-4383-a3bd-d8dff3d3a255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:18.652306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:18.801022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1eab-dc39-4fd2-97cb-d51d14bf7939 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:18.802537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-223a1eab-dc39-4fd2-97cb-d51d14bf7939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:18.911810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:18.948404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a1eab-dc39-4fd2-97cb-d51d14bf7939 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:19.062763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff6ba3-db4d-4c51-b58b-07b7ed0e0916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:19.063349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff6ba3-db4d-4c51-b58b-07b7ed0e0916 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:19.063636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff6ba3-db4d-4c51-b58b-07b7ed0e0916 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:19.063915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff6ba3-db4d-4c51-b58b-07b7ed0e0916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:19.065580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-eaff6ba3-db4d-4c51-b58b-07b7ed0e0916 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:19.141928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:19.155619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff6ba3-db4d-4c51-b58b-07b7ed0e0916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:19.157235 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff6ba3-db4d-4c51-b58b-07b7ed0e0916 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:19.175854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 710/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:20.034507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f690-22a5-4ebc-9e0c-cfa246a3c414 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:20.035082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f690-22a5-4ebc-9e0c-cfa246a3c414 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:20.035082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66f690-22a5-4ebc-9e0c-cfa246a3c414 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:20.036463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:20.041623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05364029-1fc6-4cbc-90e7-50c58574062d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:20.042103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05364029-1fc6-4cbc-90e7-50c58574062d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:20.042203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05364029-1fc6-4cbc-90e7-50c58574062d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:20.118576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-05364029-1fc6-4cbc-90e7-50c58574062d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:20.119798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-05364029-1fc6-4cbc-90e7-50c58574062d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:20.215990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:20.367548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66f690-22a5-4ebc-9e0c-cfa246a3c414 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:20.369004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-bb66f690-22a5-4ebc-9e0c-cfa246a3c414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:20.453489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:20.514508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66f690-22a5-4ebc-9e0c-cfa246a3c414 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:20.628399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c076b8-1841-4998-ab77-23432e3e4619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:20.628976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c076b8-1841-4998-ab77-23432e3e4619 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:20.629263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c076b8-1841-4998-ab77-23432e3e4619 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:20.629544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c076b8-1841-4998-ab77-23432e3e4619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:20.631243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-85c076b8-1841-4998-ab77-23432e3e4619 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:20.679005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:20.720459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c076b8-1841-4998-ab77-23432e3e4619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:20.722085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c076b8-1841-4998-ab77-23432e3e4619 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:20.740200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 717/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:21.605963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c6ebc-881c-4431-ad0a-e5b363287d4e None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:21.606525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c6ebc-881c-4431-ad0a-e5b363287d4e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:21.606525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c6ebc-881c-4431-ad0a-e5b363287d4e None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:21.607907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:21.612801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830216f-3eb8-4093-b0d0-ee91fd07c0fe None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:21.613249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830216f-3eb8-4093-b0d0-ee91fd07c0fe None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:21.613360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830216f-3eb8-4093-b0d0-ee91fd07c0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:21.688673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-5830216f-3eb8-4093-b0d0-ee91fd07c0fe None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:21.689911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5830216f-3eb8-4093-b0d0-ee91fd07c0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:21.784668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:21.933548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c6ebc-881c-4431-ad0a-e5b363287d4e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:21.935057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9c5c6ebc-881c-4431-ad0a-e5b363287d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:21.984385 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:22.084777 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c6ebc-881c-4431-ad0a-e5b363287d4e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:22.199641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eeb16-acec-47c3-8b80-9e3b4e70cca4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:22.200232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eeb16-acec-47c3-8b80-9e3b4e70cca4 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:22.200518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eeb16-acec-47c3-8b80-9e3b4e70cca4 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:22.200801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eeb16-acec-47c3-8b80-9e3b4e70cca4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:22.202495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-686eeb16-acec-47c3-8b80-9e3b4e70cca4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:22.210362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:22.293732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eeb16-acec-47c3-8b80-9e3b4e70cca4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:22.295368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eeb16-acec-47c3-8b80-9e3b4e70cca4 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:22.314178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 713/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:23.238572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be814f-6474-4a59-a0b5-57466fa10200 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:23.239159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be814f-6474-4a59-a0b5-57466fa10200 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:23.239159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be814f-6474-4a59-a0b5-57466fa10200 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:23.240509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:23.245400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2de62e-a375-4b46-974e-e4789d52531a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:23.245802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2de62e-a375-4b46-974e-e4789d52531a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:23.246003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2de62e-a375-4b46-974e-e4789d52531a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:23.323996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-8c2de62e-a375-4b46-974e-e4789d52531a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:23.325227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8c2de62e-a375-4b46-974e-e4789d52531a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:23.394194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:23.422931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:23.574913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be814f-6474-4a59-a0b5-57466fa10200 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:23.576424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c4be814f-6474-4a59-a0b5-57466fa10200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:23.594134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:23.721424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be814f-6474-4a59-a0b5-57466fa10200 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:23.806798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:23.832351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d9dff-7fbc-4b43-adca-b73d180b933e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:23.832948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d9dff-7fbc-4b43-adca-b73d180b933e service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:23.833233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d9dff-7fbc-4b43-adca-b73d180b933e service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:23.833509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d9dff-7fbc-4b43-adca-b73d180b933e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:23.835295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-eb1d9dff-7fbc-4b43-adca-b73d180b933e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:23.925172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d9dff-7fbc-4b43-adca-b73d180b933e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:23.926873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d9dff-7fbc-4b43-adca-b73d180b933e service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:23.945386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 720/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:24.794585 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05488c7-c67d-44c6-9679-b5d91d96ab70 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:24.795151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05488c7-c67d-44c6-9679-b5d91d96ab70 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:24.795151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05488c7-c67d-44c6-9679-b5d91d96ab70 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:24.796512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:24.801414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7672a-1ff6-490c-bd3d-d044d22a07ef None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:24.801842 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7672a-1ff6-490c-bd3d-d044d22a07ef None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:24.802007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7672a-1ff6-490c-bd3d-d044d22a07ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:24.876467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f3a7672a-1ff6-490c-bd3d-d044d22a07ef None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:24.877700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f3a7672a-1ff6-490c-bd3d-d044d22a07ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:24.918552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:24.974178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:25.125371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05488c7-c67d-44c6-9679-b5d91d96ab70 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:25.126954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a05488c7-c67d-44c6-9679-b5d91d96ab70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:25.271254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05488c7-c67d-44c6-9679-b5d91d96ab70 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:25.327390 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:25.386941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c51aa-ad74-4451-8487-641ab7dac553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:25.387524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c51aa-ad74-4451-8487-641ab7dac553 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:25.387823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c51aa-ad74-4451-8487-641ab7dac553 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:25.388098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c51aa-ad74-4451-8487-641ab7dac553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:25.389883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2b2c51aa-ad74-4451-8487-641ab7dac553 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:25.480163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c51aa-ad74-4451-8487-641ab7dac553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:25.481767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c51aa-ad74-4451-8487-641ab7dac553 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:25.499634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 716/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:26.382997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed0d92-cf17-48be-a3d8-79fd939a4583 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:26.383565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed0d92-cf17-48be-a3d8-79fd939a4583 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:26.383565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed0d92-cf17-48be-a3d8-79fd939a4583 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:26.384942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:26.390029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f98b3c-90ac-4aa1-b44b-f9f10db45832 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:26.390457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f98b3c-90ac-4aa1-b44b-f9f10db45832 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:26.390607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f98b3c-90ac-4aa1-b44b-f9f10db45832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:26.466353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-30f98b3c-90ac-4aa1-b44b-f9f10db45832 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:26.467589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-30f98b3c-90ac-4aa1-b44b-f9f10db45832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:26.469606 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:26.567170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:26.717930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed0d92-cf17-48be-a3d8-79fd939a4583 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:26.719491 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-efed0d92-cf17-48be-a3d8-79fd939a4583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:26.862567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed0d92-cf17-48be-a3d8-79fd939a4583 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:26.889198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:26.973770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0b859-6ff2-4f5c-b6b3-4b23f87f2323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:26.974388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b859-6ff2-4f5c-b6b3-4b23f87f2323 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:26.974673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b859-6ff2-4f5c-b6b3-4b23f87f2323 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:26.974951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0b859-6ff2-4f5c-b6b3-4b23f87f2323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:26.977125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-dee0b859-6ff2-4f5c-b6b3-4b23f87f2323 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:27.067476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0b859-6ff2-4f5c-b6b3-4b23f87f2323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:27.069575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0b859-6ff2-4f5c-b6b3-4b23f87f2323 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:27.088106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 723/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:29.031586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada6cf1-37ae-439e-a798-c286359fd6dc None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:29.032169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada6cf1-37ae-439e-a798-c286359fd6dc None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:29.032169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada6cf1-37ae-439e-a798-c286359fd6dc None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:29.033566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:29.039763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea3dee-3a92-465a-a808-eb57ca6467b5 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:29.040209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea3dee-3a92-465a-a808-eb57ca6467b5 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:29.040410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea3dee-3a92-465a-a808-eb57ca6467b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:29.080606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:29.105800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-3aea3dee-3a92-465a-a808-eb57ca6467b5 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:29.107051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3aea3dee-3a92-465a-a808-eb57ca6467b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:29.202965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:29.242413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada6cf1-37ae-439e-a798-c286359fd6dc None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:29.243962 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4ada6cf1-37ae-439e-a798-c286359fd6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:29.348258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada6cf1-37ae-439e-a798-c286359fd6dc None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:29.380058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:29.421713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea30e2-da90-42ab-b59e-b253cc5f56a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:29.422386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea30e2-da90-42ab-b59e-b253cc5f56a2 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:29.422677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea30e2-da90-42ab-b59e-b253cc5f56a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:29.422949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea30e2-da90-42ab-b59e-b253cc5f56a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:29.424638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-cfea30e2-da90-42ab-b59e-b253cc5f56a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:29.534948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea30e2-da90-42ab-b59e-b253cc5f56a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:29.536394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea30e2-da90-42ab-b59e-b253cc5f56a2 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:29.554827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 719/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:31.718513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10ff9a-df0a-460b-a4bc-10a3da6b234c None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:31.719110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10ff9a-df0a-460b-a4bc-10a3da6b234c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:31.719110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10ff9a-df0a-460b-a4bc-10a3da6b234c None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:31.720502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:31.725864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f54471-2cb9-4a39-a1bc-865ebff78ca7 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:31.726274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f54471-2cb9-4a39-a1bc-865ebff78ca7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:31.726428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f54471-2cb9-4a39-a1bc-865ebff78ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:31.790720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f2f54471-2cb9-4a39-a1bc-865ebff78ca7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:31.791929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f2f54471-2cb9-4a39-a1bc-865ebff78ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:31.805142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:31.890255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:31.928226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10ff9a-df0a-460b-a4bc-10a3da6b234c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:31.930292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8d10ff9a-df0a-460b-a4bc-10a3da6b234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:32.034394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10ff9a-df0a-460b-a4bc-10a3da6b234c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:32.047734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:32.109376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481d825-f62e-47a1-8609-455acaa35a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:32.110071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481d825-f62e-47a1-8609-455acaa35a28 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:32.110358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481d825-f62e-47a1-8609-455acaa35a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:32.110636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481d825-f62e-47a1-8609-455acaa35a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:32.112293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e481d825-f62e-47a1-8609-455acaa35a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:32.222244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481d825-f62e-47a1-8609-455acaa35a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:32.223696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481d825-f62e-47a1-8609-455acaa35a28 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:32.242504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 726/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:34.628893 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b1f0f-74dc-4f1f-a91e-910357a5b959 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:34.629469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b1f0f-74dc-4f1f-a91e-910357a5b959 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:34.629469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b1f0f-74dc-4f1f-a91e-910357a5b959 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:34.630945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:34.636161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510fdd1-b077-46ce-847f-fdeed66c6441 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:34.636590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510fdd1-b077-46ce-847f-fdeed66c6441 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:34.636731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510fdd1-b077-46ce-847f-fdeed66c6441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:34.683435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:34.700351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b510fdd1-b077-46ce-847f-fdeed66c6441 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:34.701561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b510fdd1-b077-46ce-847f-fdeed66c6441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:34.798799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:34.838555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b1f0f-74dc-4f1f-a91e-910357a5b959 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:34.840140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b65b1f0f-74dc-4f1f-a91e-910357a5b959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:34.942950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b1f0f-74dc-4f1f-a91e-910357a5b959 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:34.982806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:35.016940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4dd7c-fa17-47f1-af2c-a23c57125f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:35.017592 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4dd7c-fa17-47f1-af2c-a23c57125f0b service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:35.017909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4dd7c-fa17-47f1-af2c-a23c57125f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:35.018189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4dd7c-fa17-47f1-af2c-a23c57125f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:35.019837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6db4dd7c-fa17-47f1-af2c-a23c57125f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:35.131554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4dd7c-fa17-47f1-af2c-a23c57125f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:35.133006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4dd7c-fa17-47f1-af2c-a23c57125f0b service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:35.151507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 722/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:37.295209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fadcb5-fccd-432e-b158-4e777b2f4148 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:37.295661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fadcb5-fccd-432e-b158-4e777b2f4148 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:37.295759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fadcb5-fccd-432e-b158-4e777b2f4148 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:37.297096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:37.302423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3cacd-e7b0-4837-a6c5-fb9aae83808d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:37.302834 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3cacd-e7b0-4837-a6c5-fb9aae83808d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:37.302998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3cacd-e7b0-4837-a6c5-fb9aae83808d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:37.367668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-52c3cacd-e7b0-4837-a6c5-fb9aae83808d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:37.368880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-52c3cacd-e7b0-4837-a6c5-fb9aae83808d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:37.390039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:37.468321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:37.506221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fadcb5-fccd-432e-b158-4e777b2f4148 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:37.508273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-92fadcb5-fccd-432e-b158-4e777b2f4148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:37.612578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fadcb5-fccd-432e-b158-4e777b2f4148 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:37.632595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:37.688025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9dc84-c810-47f3-ab9a-694b6ead9b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:37.688676 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9dc84-c810-47f3-ab9a-694b6ead9b4e service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:37.688957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9dc84-c810-47f3-ab9a-694b6ead9b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:37.689227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9dc84-c810-47f3-ab9a-694b6ead9b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:37.690912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0ef9dc84-c810-47f3-ab9a-694b6ead9b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:37.796330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9dc84-c810-47f3-ab9a-694b6ead9b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:37.797881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9dc84-c810-47f3-ab9a-694b6ead9b4e service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:37.816129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 729/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:39.898378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b4d3-d2f5-407f-bf8a-c7dbd4e4cbce None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:39.898946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b4d3-d2f5-407f-bf8a-c7dbd4e4cbce None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:39.898946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906b4d3-d2f5-407f-bf8a-c7dbd4e4cbce None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:39.900303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:39.905473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb52c53-8c5f-404b-b15d-22dc69b43947 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:39.905951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb52c53-8c5f-404b-b15d-22dc69b43947 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:39.906073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb52c53-8c5f-404b-b15d-22dc69b43947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:39.969444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:39.972116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-4eb52c53-8c5f-404b-b15d-22dc69b43947 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:39.973339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4eb52c53-8c5f-404b-b15d-22dc69b43947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:40.071420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:40.109441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906b4d3-d2f5-407f-bf8a-c7dbd4e4cbce None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:40.111190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d906b4d3-d2f5-407f-bf8a-c7dbd4e4cbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:40.215457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906b4d3-d2f5-407f-bf8a-c7dbd4e4cbce None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:40.264639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:40.288615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d496df9-fb65-4e85-82b6-1c300b0b8253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:40.289930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d496df9-fb65-4e85-82b6-1c300b0b8253 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:40.289930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d496df9-fb65-4e85-82b6-1c300b0b8253 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:40.289930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d496df9-fb65-4e85-82b6-1c300b0b8253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:40.291539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6d496df9-fb65-4e85-82b6-1c300b0b8253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:40.399457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d496df9-fb65-4e85-82b6-1c300b0b8253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:40.400809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d496df9-fb65-4e85-82b6-1c300b0b8253 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:40.419434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 725/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:48:42.497957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f40585-ef18-40ee-aa77-ead4df07f031 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:42.498529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f40585-ef18-40ee-aa77-ead4df07f031 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:42.498529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f40585-ef18-40ee-aa77-ead4df07f031 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:42.499913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:42.505217 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675a201-f833-411f-b28f-0d3369bfe76a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:42.505664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675a201-f833-411f-b28f-0d3369bfe76a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:42.505837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675a201-f833-411f-b28f-0d3369bfe76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:42.571528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-9675a201-f833-411f-b28f-0d3369bfe76a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:42.572742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9675a201-f833-411f-b28f-0d3369bfe76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:42.602009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:42.670607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:42.709253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f40585-ef18-40ee-aa77-ead4df07f031 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:42.710854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d6f40585-ef18-40ee-aa77-ead4df07f031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:42.813776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f40585-ef18-40ee-aa77-ead4df07f031 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:42.842450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:42.887595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9f43c-1a3f-480b-ad20-4fc2ef589f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:42.888255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9f43c-1a3f-480b-ad20-4fc2ef589f34 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:42.888541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9f43c-1a3f-480b-ad20-4fc2ef589f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:42.888822 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9f43c-1a3f-480b-ad20-4fc2ef589f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:42.890522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d6c9f43c-1a3f-480b-ad20-4fc2ef589f34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:42.998471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9f43c-1a3f-480b-ad20-4fc2ef589f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:42.999900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9f43c-1a3f-480b-ad20-4fc2ef589f34 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:43.018005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 732/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:45.098550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa4117-a9d2-4566-b505-992157b4f46d None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:45.099070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa4117-a9d2-4566-b505-992157b4f46d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:45.099192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa4117-a9d2-4566-b505-992157b4f46d None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:45.101027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:45.106282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7efbe-98db-46d6-8a04-5095811e548a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:45.106662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7efbe-98db-46d6-8a04-5095811e548a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:45.106854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7efbe-98db-46d6-8a04-5095811e548a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:45.171157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-51a7efbe-98db-46d6-8a04-5095811e548a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:45.172375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-51a7efbe-98db-46d6-8a04-5095811e548a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:45.181610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:45.269070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:45.306855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa4117-a9d2-4566-b505-992157b4f46d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:45.308376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d9aa4117-a9d2-4566-b505-992157b4f46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:45.412971 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa4117-a9d2-4566-b505-992157b4f46d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:45.470655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:45.489001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5947d-b39e-4b95-a309-4d4e4c3a0907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:45.489649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5947d-b39e-4b95-a309-4d4e4c3a0907 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:45.489979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5947d-b39e-4b95-a309-4d4e4c3a0907 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:45.490255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5947d-b39e-4b95-a309-4d4e4c3a0907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:45.492453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ecc5947d-b39e-4b95-a309-4d4e4c3a0907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:45.601341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5947d-b39e-4b95-a309-4d4e4c3a0907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:45.602810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5947d-b39e-4b95-a309-4d4e4c3a0907 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:45.621722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 728/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:47.681425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d95ffa-4781-41aa-b0de-b1c2917fb288 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:47.682018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d95ffa-4781-41aa-b0de-b1c2917fb288 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:47.682018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d95ffa-4781-41aa-b0de-b1c2917fb288 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:47.683472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:47.688748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16538a3-a15e-4e42-b4aa-3706c34906a8 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:47.689175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16538a3-a15e-4e42-b4aa-3706c34906a8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:47.689350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16538a3-a15e-4e42-b4aa-3706c34906a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:47.762660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e16538a3-a15e-4e42-b4aa-3706c34906a8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:47.763862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e16538a3-a15e-4e42-b4aa-3706c34906a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:47.802027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:47.860150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:47.898280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d95ffa-4781-41aa-b0de-b1c2917fb288 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:47.899853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a1d95ffa-4781-41aa-b0de-b1c2917fb288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:48.001792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d95ffa-4781-41aa-b0de-b1c2917fb288 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:48.040505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:48.076585 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdb188-edcd-4e62-b14e-c08a14829fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:48.077241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdb188-edcd-4e62-b14e-c08a14829fdb service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:48.077526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdb188-edcd-4e62-b14e-c08a14829fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:48.077801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdb188-edcd-4e62-b14e-c08a14829fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:48.079468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-49fdb188-edcd-4e62-b14e-c08a14829fdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:48.187983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdb188-edcd-4e62-b14e-c08a14829fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:48.189426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdb188-edcd-4e62-b14e-c08a14829fdb service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:48.209277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 735/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:50.327474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426abe-413e-4db7-a21f-056a5c5c3d57 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:50.328032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426abe-413e-4db7-a21f-056a5c5c3d57 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:50.328032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426abe-413e-4db7-a21f-056a5c5c3d57 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:50.329526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:50.335428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000ad1a-5904-48be-bc8d-870d3cb9300f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:50.335918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000ad1a-5904-48be-bc8d-870d3cb9300f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:50.336046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000ad1a-5904-48be-bc8d-870d3cb9300f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:50.401527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-6000ad1a-5904-48be-bc8d-870d3cb9300f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:50.402781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6000ad1a-5904-48be-bc8d-870d3cb9300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:50.420778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:50.500474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:50.537553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74426abe-413e-4db7-a21f-056a5c5c3d57 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:50.539185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-74426abe-413e-4db7-a21f-056a5c5c3d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:50.644521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74426abe-413e-4db7-a21f-056a5c5c3d57 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:50.718773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:50.726584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8cd97b-4600-4f84-8da0-1ec2d4079945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:50.727335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cd97b-4600-4f84-8da0-1ec2d4079945 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:50.727663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cd97b-4600-4f84-8da0-1ec2d4079945 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:50.728471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8cd97b-4600-4f84-8da0-1ec2d4079945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:50.730036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7a8cd97b-4600-4f84-8da0-1ec2d4079945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:50.854258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cd97b-4600-4f84-8da0-1ec2d4079945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:50.855939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8cd97b-4600-4f84-8da0-1ec2d4079945 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:50.876837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 731/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:48:52.934020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f3080-bf2b-4c3d-8696-0b6a98fa3eee None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:52.934577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f3080-bf2b-4c3d-8696-0b6a98fa3eee None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:52.934577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f3080-bf2b-4c3d-8696-0b6a98fa3eee None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:52.935982 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:48:52.941268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca699c0f-5a6e-4d62-b752-79fa75d73783 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:52.941681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca699c0f-5a6e-4d62-b752-79fa75d73783 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:52.941852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca699c0f-5a6e-4d62-b752-79fa75d73783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:53.008408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ca699c0f-5a6e-4d62-b752-79fa75d73783 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:48:53.009627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ca699c0f-5a6e-4d62-b752-79fa75d73783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:53.058941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:53.108329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:48:53.145328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f3080-bf2b-4c3d-8696-0b6a98fa3eee None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:48:53.146912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-912f3080-bf2b-4c3d-8696-0b6a98fa3eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:53.251310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f3080-bf2b-4c3d-8696-0b6a98fa3eee None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:48:53.299974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:48:53.326748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89adee8-c801-4fb1-b9bd-0f97b681b291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:48:53.327408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89adee8-c801-4fb1-b9bd-0f97b681b291 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:48:53.327695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89adee8-c801-4fb1-b9bd-0f97b681b291 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:48:53.327977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89adee8-c801-4fb1-b9bd-0f97b681b291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:48:53.329632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f89adee8-c801-4fb1-b9bd-0f97b681b291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:48:53.437664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89adee8-c801-4fb1-b9bd-0f97b681b291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:48:53.439071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89adee8-c801-4fb1-b9bd-0f97b681b291 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:48:53.457206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 738/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:48:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:49:03.143394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=22 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:03.454939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=66 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:13.263910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69021ec-deac-456e-a0e1-030b90c3cbf8 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:13.264465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69021ec-deac-456e-a0e1-030b90c3cbf8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:13.264465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69021ec-deac-456e-a0e1-030b90c3cbf8 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:13.265889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 733/1471] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:49:13.271690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551a284-8fcc-437e-85a2-5948b1b978ab None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:13.272136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551a284-8fcc-437e-85a2-5948b1b978ab None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:13.272278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551a284-8fcc-437e-85a2-5948b1b978ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:13.338185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-4551a284-8fcc-437e-85a2-5948b1b978ab None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:49:13.339402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4551a284-8fcc-437e-85a2-5948b1b978ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:13.435208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 739/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:49:13.518622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32900088-df22-4e7a-a62d-248bf4876a5a None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:13.518916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32900088-df22-4e7a-a62d-248bf4876a5a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:13.519197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32900088-df22-4e7a-a62d-248bf4876a5a None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:13.520567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 734/1473] 10.222.0.47 () {54 vars in 962 bytes} [Thu Jul 14 15:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:49:13.525250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1033fd7-20d7-4d23-a30c-700b7f99bc0d None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:13.525540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1033fd7-20d7-4d23-a30c-700b7f99bc0d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:13.525863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1033fd7-20d7-4d23-a30c-700b7f99bc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:13.599246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b1033fd7-20d7-4d23-a30c-700b7f99bc0d None None] MFA Rules not processed for user `5eeda3264191409eba35ffbb908770a6`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:49:13.600890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b1033fd7-20d7-4d23-a30c-700b7f99bc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:13.621590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:13.675902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 740/1474] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Jul 14 15:49:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:49:13.684084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32900088-df22-4e7a-a62d-248bf4876a5a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:49:13.685608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-32900088-df22-4e7a-a62d-248bf4876a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:13.789404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32900088-df22-4e7a-a62d-248bf4876a5a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:49:13.862372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9da405-d7d7-4f02-8885-33ac4b46dcb0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5eeda3264191409eba35ffbb908770a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:49:13.862954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da405-d7d7-4f02-8885-33ac4b46dcb0 service glance] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:13.863254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da405-d7d7-4f02-8885-33ac4b46dcb0 service glance] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:13.863523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9da405-d7d7-4f02-8885-33ac4b46dcb0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:13.865207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0a9da405-d7d7-4f02-8885-33ac4b46dcb0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:13.919812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:13.973883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9da405-d7d7-4f02-8885-33ac4b46dcb0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:49:13.975882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9da405-d7d7-4f02-8885-33ac4b46dcb0 service glance] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:49:13.994015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 735/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:49:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:49:16.091528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63f56-06d2-413e-bee2-abb79cddca80 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:16.092077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63f56-06d2-413e-bee2-abb79cddca80 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:16.092077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63f56-06d2-413e-bee2-abb79cddca80 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:16.093481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 741/1476] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:49:16.098857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc32bee-c3e3-4f52-82bc-a604e571011d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:16.099313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc32bee-c3e3-4f52-82bc-a604e571011d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:16.099453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc32bee-c3e3-4f52-82bc-a604e571011d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:16.162810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-8fc32bee-c3e3-4f52-82bc-a604e571011d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:49:16.164029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8fc32bee-c3e3-4f52-82bc-a604e571011d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:16.259412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 736/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:49:16.282512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c63f56-06d2-413e-bee2-abb79cddca80 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:49:16.284050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-44c63f56-06d2-413e-bee2-abb79cddca80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:16.408460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c63f56-06d2-413e-bee2-abb79cddca80 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:49:16.455934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:16.501338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de0751-79d7-44ed-a2a1-990c929a700e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:49:16.502036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de0751-79d7-44ed-a2a1-990c929a700e admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:16.502322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de0751-79d7-44ed-a2a1-990c929a700e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:16.502600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de0751-79d7-44ed-a2a1-990c929a700e admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:16.503338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de0751-79d7-44ed-a2a1-990c929a700e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:49:16.505115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING py.warnings [None req-52de0751-79d7-44ed-a2a1-990c929a700e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:49:16.505115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: warnings.warn(msg) Jul 14 15:49:16.505115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: Jul 14 15:49:16.506793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de0751-79d7-44ed-a2a1-990c929a700e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:49:16.526451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 742/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 14 15:49:16 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:49:18.570620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb147b-09d2-41e4-891b-6c7e737ede3a None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:18.571188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb147b-09d2-41e4-891b-6c7e737ede3a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:18.571188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb147b-09d2-41e4-891b-6c7e737ede3a None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:18.572553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 737/1479] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:49:18.577958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039980f3-ccef-4c30-a605-9324a2b7e631 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:18.578411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039980f3-ccef-4c30-a605-9324a2b7e631 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:18.578543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039980f3-ccef-4c30-a605-9324a2b7e631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:18.642393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-039980f3-ccef-4c30-a605-9324a2b7e631 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:49:18.643617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-039980f3-ccef-4c30-a605-9324a2b7e631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:18.739132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 743/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:49:18.782409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb147b-09d2-41e4-891b-6c7e737ede3a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:49:18.783927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-dffb147b-09d2-41e4-891b-6c7e737ede3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:18.793673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:18.929865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb147b-09d2-41e4-891b-6c7e737ede3a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:49:19.009984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:19.044588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7a16b-3a39-4059-9809-1601ad75e5a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:49:19.045253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7a16b-3a39-4059-9809-1601ad75e5a9 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:19.045541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7a16b-3a39-4059-9809-1601ad75e5a9 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:19.045841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7a16b-3a39-4059-9809-1601ad75e5a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:19.047540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-15c7a16b-3a39-4059-9809-1601ad75e5a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:19.254306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7a16b-3a39-4059-9809-1601ad75e5a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:49:19.256471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7a16b-3a39-4059-9809-1601ad75e5a9 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:49:19.274707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 738/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:49:21.457783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9c672-8cfc-4752-a652-756f7ffd8b74 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:21.458384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9c672-8cfc-4752-a652-756f7ffd8b74 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:21.458384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9c672-8cfc-4752-a652-756f7ffd8b74 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:21.460289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 744/1482] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:49:21.465467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8c074-97ea-4871-9fa1-c1b1914b0215 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:21.465898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8c074-97ea-4871-9fa1-c1b1914b0215 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:21.466083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8c074-97ea-4871-9fa1-c1b1914b0215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:21.531376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-4be8c074-97ea-4871-9fa1-c1b1914b0215 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:49:21.532584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4be8c074-97ea-4871-9fa1-c1b1914b0215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:21.579668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:21.629099 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 739/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:49:21.665663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9c672-8cfc-4752-a652-756f7ffd8b74 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:49:21.667230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-22e9c672-8cfc-4752-a652-756f7ffd8b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:21.690628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:21.772073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9c672-8cfc-4752-a652-756f7ffd8b74 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:49:21.843972 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f75bee-4dcd-434a-9005-809cc88c0823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:49:21.844566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f75bee-4dcd-434a-9005-809cc88c0823 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:21.844849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f75bee-4dcd-434a-9005-809cc88c0823 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:21.845126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f75bee-4dcd-434a-9005-809cc88c0823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:21.846792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f0f75bee-4dcd-434a-9005-809cc88c0823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:21.920049 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:21.955876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f75bee-4dcd-434a-9005-809cc88c0823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:49:21.957318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f75bee-4dcd-434a-9005-809cc88c0823 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:49:21.977886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 745/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:49:24.066266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a81e83-2255-4b3a-bf64-212b9451c766 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:24.066766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a81e83-2255-4b3a-bf64-212b9451c766 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:24.066865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a81e83-2255-4b3a-bf64-212b9451c766 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:24.068759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 740/1485] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:49:24.074024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fd510-9279-462f-89ce-7df7157fb896 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:24.074464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fd510-9279-462f-89ce-7df7157fb896 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:24.074577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fd510-9279-462f-89ce-7df7157fb896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:24.137184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-1b5fd510-9279-462f-89ce-7df7157fb896 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:49:24.139211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1b5fd510-9279-462f-89ce-7df7157fb896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:24.244939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 746/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:49:24.283284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a81e83-2255-4b3a-bf64-212b9451c766 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:49:24.284809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c4a81e83-2255-4b3a-bf64-212b9451c766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:24.388974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a81e83-2255-4b3a-bf64-212b9451c766 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:49:24.458821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:24.460539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b61cf7-8baa-4519-8bf2-cbbb2f45ae39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:49:24.461181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b61cf7-8baa-4519-8bf2-cbbb2f45ae39 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:24.461457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b61cf7-8baa-4519-8bf2-cbbb2f45ae39 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:24.461723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b61cf7-8baa-4519-8bf2-cbbb2f45ae39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:24.463375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b0b61cf7-8baa-4519-8bf2-cbbb2f45ae39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:24.573400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b61cf7-8baa-4519-8bf2-cbbb2f45ae39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:49:24.574791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b61cf7-8baa-4519-8bf2-cbbb2f45ae39 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:49:24.593105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 741/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:49:26.783840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce753b-023f-4917-b7e6-1fc902955361 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:26.784420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce753b-023f-4917-b7e6-1fc902955361 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:26.784420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce753b-023f-4917-b7e6-1fc902955361 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:26.785784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 747/1488] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:49:26.791338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01d76-6962-4f00-9ac3-fab2e72bc089 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:26.791758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01d76-6962-4f00-9ac3-fab2e72bc089 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:26.791931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a01d76-6962-4f00-9ac3-fab2e72bc089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:26.856331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a9a01d76-6962-4f00-9ac3-fab2e72bc089 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:49:26.857535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a9a01d76-6962-4f00-9ac3-fab2e72bc089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:26.909664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:26.955177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 742/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:49:26.995733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce753b-023f-4917-b7e6-1fc902955361 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:49:26.997262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-89ce753b-023f-4917-b7e6-1fc902955361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:27.033093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:27.100735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce753b-023f-4917-b7e6-1fc902955361 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:49:27.173125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915132e9-d91a-4540-9b38-a79bee88caaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:49:27.173785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915132e9-d91a-4540-9b38-a79bee88caaf service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:27.174103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915132e9-d91a-4540-9b38-a79bee88caaf service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:27.174386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915132e9-d91a-4540-9b38-a79bee88caaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:27.176037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-915132e9-d91a-4540-9b38-a79bee88caaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:27.257427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:27.286229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915132e9-d91a-4540-9b38-a79bee88caaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:49:27.287769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915132e9-d91a-4540-9b38-a79bee88caaf service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:49:27.306752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 748/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:49:29.680891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77472fe-d6da-4949-bf18-5ae8b897c92c None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:29.681504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77472fe-d6da-4949-bf18-5ae8b897c92c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:29.681504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77472fe-d6da-4949-bf18-5ae8b897c92c None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:29.682782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 743/1491] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 14 15:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:49:29.688124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951f398-e28b-414a-bb1d-d39d2e193ef2 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:29.688540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951f398-e28b-414a-bb1d-d39d2e193ef2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:29.688673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951f398-e28b-414a-bb1d-d39d2e193ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:29.758836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e951f398-e28b-414a-bb1d-d39d2e193ef2 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:49:29.760078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e951f398-e28b-414a-bb1d-d39d2e193ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:29.857471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 749/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 14 15:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:49:29.896435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77472fe-d6da-4949-bf18-5ae8b897c92c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:49:29.898019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e77472fe-d6da-4949-bf18-5ae8b897c92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:30.003112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77472fe-d6da-4949-bf18-5ae8b897c92c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:49:30.077420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4f3c1-4f1f-4313-a7e6-258aef13d1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:49:30.078105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4f3c1-4f1f-4313-a7e6-258aef13d1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:49:30.078392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4f3c1-4f1f-4313-a7e6-258aef13d1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:49:30.078666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4f3c1-4f1f-4313-a7e6-258aef13d1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:49:30.080311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f7e4f3c1-4f1f-4313-a7e6-258aef13d1c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:49:30.089316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:30.189166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4f3c1-4f1f-4313-a7e6-258aef13d1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:49:30.190640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4f3c1-4f1f-4313-a7e6-258aef13d1c0 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:49:30.208869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 744/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:49:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:49:39.854788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=82 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:49:40.206120 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=50 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:06.389369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccf641-82fb-4b25-b833-3aa20ac1bf52 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:06.389997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccf641-82fb-4b25-b833-3aa20ac1bf52 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:06.389997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccf641-82fb-4b25-b833-3aa20ac1bf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:06.462681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-84ccf641-82fb-4b25-b833-3aa20ac1bf52 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:51:06.463902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-84ccf641-82fb-4b25-b833-3aa20ac1bf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:06.559310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 750/1494] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:51:06.568043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb985-58a5-4f79-8adf-85d2ad7a7853 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:06.568503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb985-58a5-4f79-8adf-85d2ad7a7853 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:06.568625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb985-58a5-4f79-8adf-85d2ad7a7853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:06.646368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a6feb985-58a5-4f79-8adf-85d2ad7a7853 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:51:06.647510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a6feb985-58a5-4f79-8adf-85d2ad7a7853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:06.744157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 745/1495] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:51:06.748930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccf641-82fb-4b25-b833-3aa20ac1bf52 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:06.750223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-84ccf641-82fb-4b25-b833-3aa20ac1bf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:06.761972 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccf641-82fb-4b25-b833-3aa20ac1bf52 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:06.823374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:06.857145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:06.857844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:06.858127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:06.858402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:06.859126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:51:06.859430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:06.860439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING py.warnings [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:51:06.860439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: warnings.warn(msg) Jul 14 15:51:06.860439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: Jul 14 15:51:06.861373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:06.872886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 751/1496] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:51:06.881596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feb985-58a5-4f79-8adf-85d2ad7a7853 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:06.890727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feb985-58a5-4f79-8adf-85d2ad7a7853 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:06.952199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:06.985056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:06.985715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:06.986034 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:06.986312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:06.987018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:06.990053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WARNING py.warnings [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:51:06.990053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: warnings.warn(msg) Jul 14 15:51:06.990053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: Jul 14 15:51:06.991242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:07.135983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 746/1497] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:51:06 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:51:07.140269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:07.141861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:07.175717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79217c-b31f-4f27-a6ee-8f4214ef7ac4 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:07.271274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:07.271948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:07.272233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:07.272506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:07.273192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:07.274234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING py.warnings [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:51:07.274234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: warnings.warn(msg) Jul 14 15:51:07.274234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: Jul 14 15:51:07.275456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:07.411728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 752/1498] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:51:07 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:51:07.415638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:07.417223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:07.481507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e022b9-d6a6-4d87-a9bd-2cd932515744 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:07.567155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:07.577709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:07.578421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:07.578705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:07.579863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:07.580650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:07.582309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WARNING py.warnings [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:51:07.582309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: warnings.warn(msg) Jul 14 15:51:07.582309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: Jul 14 15:51:07.583191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:07.595728 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 747/1499] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:51:07.599879 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:07.601413 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:07.613195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db73b18-3668-4c2b-9b82-79dc1fa90600 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:07.638321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:07.708133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e5b3f-1411-4eae-bf41-43436d343124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:07.708809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e5b3f-1411-4eae-bf41-43436d343124 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:07.709095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e5b3f-1411-4eae-bf41-43436d343124 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:07.709371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e5b3f-1411-4eae-bf41-43436d343124 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:07.710131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e5b3f-1411-4eae-bf41-43436d343124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:07.711143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING py.warnings [None req-3f7e5b3f-1411-4eae-bf41-43436d343124 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:51:07.711143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: warnings.warn(msg) Jul 14 15:51:07.711143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: Jul 14 15:51:07.712061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e5b3f-1411-4eae-bf41-43436d343124 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:07.724519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 753/1500] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:51:07.728676 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:07.730239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:07.764583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1706e-40b7-492d-8c49-6976c575981e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:07.876619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2ae77-44f2-4773-af21-13dea6b6d92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:07.877430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2ae77-44f2-4773-af21-13dea6b6d92f admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:07.877719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2ae77-44f2-4773-af21-13dea6b6d92f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:07.878023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2ae77-44f2-4773-af21-13dea6b6d92f admin admin] PATH_INFO: `/v3/projects/7e34bb6407bb4989986628b47e98751b/users/3693e25b3ec3446aa0415befd44a30a7/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:07.900108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2ae77-44f2-4773-af21-13dea6b6d92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e34bb6407bb4989986628b47e98751b, user_id=3693e25b3ec3446aa0415befd44a30a7, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:07.901150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WARNING py.warnings [None req-18e2ae77-44f2-4773-af21-13dea6b6d92f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:51:07.901150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: warnings.warn(msg) Jul 14 15:51:07.901150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: Jul 14 15:51:07.903124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2ae77-44f2-4773-af21-13dea6b6d92f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:07.986885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 748/1501] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:51:07 2022] PUT /identity/v3/projects/7e34bb6407bb4989986628b47e98751b/users/3693e25b3ec3446aa0415befd44a30a7/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:51:07.994762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc292d-4a46-4282-a1af-29b0f3a0d14f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:07.995058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc292d-4a46-4282-a1af-29b0f3a0d14f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:07.995342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc292d-4a46-4282-a1af-29b0f3a0d14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:08.054447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-24cc292d-4a46-4282-a1af-29b0f3a0d14f None None] MFA Rules not processed for user `3693e25b3ec3446aa0415befd44a30a7`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:51:08.055698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-24cc292d-4a46-4282-a1af-29b0f3a0d14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:08.134061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 754/1502] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:51:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:51:08.147051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f198c-825f-4520-92cd-20e7e43a5f04 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:08.147343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f198c-825f-4520-92cd-20e7e43a5f04 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:08.147639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f198c-825f-4520-92cd-20e7e43a5f04 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:08.149010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 749/1503] 10.222.0.47 () {54 vars in 963 bytes} [Thu Jul 14 15:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:51:08.153979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bba6be-3657-44d5-bf2a-904f8c151287 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:08.154294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bba6be-3657-44d5-bf2a-904f8c151287 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:08.154572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bba6be-3657-44d5-bf2a-904f8c151287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:08.185085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:08.228633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a6bba6be-3657-44d5-bf2a-904f8c151287 None None] MFA Rules not processed for user `3d54d684efce44b99f3235e80beb1c56`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:51:08.229860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a6bba6be-3657-44d5-bf2a-904f8c151287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:08.305581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 755/1504] 10.222.0.47 () {58 vars in 1060 bytes} [Thu Jul 14 15:51:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:51:08.313222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f198c-825f-4520-92cd-20e7e43a5f04 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:08.314749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f34f198c-825f-4520-92cd-20e7e43a5f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:08.364107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:08.417775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f198c-825f-4520-92cd-20e7e43a5f04 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:08.491119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:08.491771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:08.492057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:08.492338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:08.494023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:08.591956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:08.593399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:08.596964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:08.612819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 750/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:51:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:51:08.654249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef367-6db7-47f9-9126-bbccee021be0 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:08.654654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef367-6db7-47f9-9126-bbccee021be0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:08.654805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef367-6db7-47f9-9126-bbccee021be0 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:08.656102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 756/1506] 10.222.0.47 () {52 vars in 829 bytes} [Thu Jul 14 15:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:51:08.660277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:08.661503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:08.760062 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552189d8-6fee-406b-9208-406f560a4876 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:08.854599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:08.855474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:08.855758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:08.856033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] PATH_INFO: `/v3/users/3693e25b3ec3446aa0415befd44a30a7` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:08.868307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3693e25b3ec3446aa0415befd44a30a7)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:08.869365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WARNING py.warnings [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:51:08.869365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: warnings.warn(msg) Jul 14 15:51:08.869365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: Jul 14 15:51:08.870613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:08.880691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:09.101923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3693e25b3ec3446aa0415befd44a30a7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:09.173997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3693e25b3ec3446aa0415befd44a30a7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:09.174428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3693e25b3ec3446aa0415befd44a30a7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:09.187154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3693e25b3ec3446aa0415befd44a30a7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:09.187683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3693e25b3ec3446aa0415befd44a30a7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:09.212752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 751/1507] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:51:08 2022] DELETE /identity/v3/users/3693e25b3ec3446aa0415befd44a30a7 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:51:09.217211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a6256-8e1d-4528-91d6-fc3a7264de36 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:09.217527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a6256-8e1d-4528-91d6-fc3a7264de36 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:09.217847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a6256-8e1d-4528-91d6-fc3a7264de36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:09.391322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7e0a6256-8e1d-4528-91d6-fc3a7264de36 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:51:09.392542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7e0a6256-8e1d-4528-91d6-fc3a7264de36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:09.489474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:09.501661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 757/1508] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:51:09.511986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:09.513225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:09.614543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d42f6-6f67-4b7b-a7f4-d9e3e29d8563 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:09.669462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:09.688156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60e520-5c93-4376-a640-c4c8b7b26bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:09.694157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60e520-5c93-4376-a640-c4c8b7b26bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:09.694157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60e520-5c93-4376-a640-c4c8b7b26bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:09.694157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60e520-5c93-4376-a640-c4c8b7b26bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:09.694157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7b60e520-5c93-4376-a640-c4c8b7b26bc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:09.804666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60e520-5c93-4376-a640-c4c8b7b26bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:09.806150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60e520-5c93-4376-a640-c4c8b7b26bc2 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:09.825495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 752/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:51:10.424454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a6256-8e1d-4528-91d6-fc3a7264de36 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:51:10.425710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7e0a6256-8e1d-4528-91d6-fc3a7264de36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:51:10.437394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a6256-8e1d-4528-91d6-fc3a7264de36 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:51:10.535273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:51:10.536140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:51:10.536422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:51:10.536698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] PATH_INFO: `/v3/projects/7e34bb6407bb4989986628b47e98751b` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:51:10.547205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e34bb6407bb4989986628b47e98751b)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:51:10.548236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING py.warnings [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:51:10.548236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: warnings.warn(msg) Jul 14 15:51:10.548236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: Jul 14 15:51:10.549429 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:51:10.666457 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e34bb6407bb4989986628b47e98751b', 'request_id': 'req-ecc28028-4af3-4c63-8d0a-a54002044e53'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:10.672687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e34bb6407bb4989986628b47e98751b', 'request_id': 'req-ecc28028-4af3-4c63-8d0a-a54002044e53'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:10.770852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e34bb6407bb4989986628b47e98751b', 'request_id': 'req-ecc28028-4af3-4c63-8d0a-a54002044e53'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:10.803408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] The token cache is being invalidate because project 7e34bb6407bb4989986628b47e98751b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:51:10.803692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:10.804490 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ecc28028-4af3-4c63-8d0a-a54002044e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:51:10.805875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 758/1510] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:51:10 2022] DELETE /identity/v3/projects/7e34bb6407bb4989986628b47e98751b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:51:19.823078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=62 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:51:20.801373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:52:53.827096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d1384-4847-4440-9707-e9c745c805e7 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:53.827833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d1384-4847-4440-9707-e9c745c805e7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:53.827833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d1384-4847-4440-9707-e9c745c805e7 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:53.829028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:52:53.834626 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785ffd6-4c0f-4050-aa6d-7566ebd94947 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:53.835142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785ffd6-4c0f-4050-aa6d-7566ebd94947 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:53.835271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785ffd6-4c0f-4050-aa6d-7566ebd94947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:53.913738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7785ffd6-4c0f-4050-aa6d-7566ebd94947 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:52:53.914990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7785ffd6-4c0f-4050-aa6d-7566ebd94947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:52:54.016485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:52:54.170017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d1384-4847-4440-9707-e9c745c805e7 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:52:54.172149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2d7d1384-4847-4440-9707-e9c745c805e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:52:54.292278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d1384-4847-4440-9707-e9c745c805e7 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:52:54.297867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52042668-c957-41a5-96e2-fa12591bfea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:52:54.299093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52042668-c957-41a5-96e2-fa12591bfea7 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:54.299377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52042668-c957-41a5-96e2-fa12591bfea7 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:54.299662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52042668-c957-41a5-96e2-fa12591bfea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:54.308030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52042668-c957-41a5-96e2-fa12591bfea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:52:54.309613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52042668-c957-41a5-96e2-fa12591bfea7 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:52:54.319698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 754/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:52:55.150312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8617dce-e426-42cc-b646-bd6c3d1393e3 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:55.151174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8617dce-e426-42cc-b646-bd6c3d1393e3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:55.151174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8617dce-e426-42cc-b646-bd6c3d1393e3 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:55.152377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:52:55.157448 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805278af-89e2-40d3-b4f2-0ab6201792f0 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:55.158555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805278af-89e2-40d3-b4f2-0ab6201792f0 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:55.158666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805278af-89e2-40d3-b4f2-0ab6201792f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:55.176800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-805278af-89e2-40d3-b4f2-0ab6201792f0 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:52:55.178299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-805278af-89e2-40d3-b4f2-0ab6201792f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:52:55.186123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:52:55.336727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8617dce-e426-42cc-b646-bd6c3d1393e3 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:52:55.351558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8617dce-e426-42cc-b646-bd6c3d1393e3 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:52:55.356805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4af90-5c7b-425b-9f6e-e88eebc37c47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:52:55.357439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4af90-5c7b-425b-9f6e-e88eebc37c47 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:55.357717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4af90-5c7b-425b-9f6e-e88eebc37c47 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:55.358020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4af90-5c7b-425b-9f6e-e88eebc37c47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:55.365962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4af90-5c7b-425b-9f6e-e88eebc37c47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:52:55.367529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4af90-5c7b-425b-9f6e-e88eebc37c47 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:52:55.376937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 761/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:52:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:52:56.725932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b67f0-c4f3-4dd3-9239-8bd0c1e3fa6a None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:56.726506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b67f0-c4f3-4dd3-9239-8bd0c1e3fa6a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:56.726506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b67f0-c4f3-4dd3-9239-8bd0c1e3fa6a None None] PATH_INFO: `/v3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:56.727867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 756/1517] 10.222.0.67 () {54 vars in 925 bytes} [Thu Jul 14 15:52:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:52:56.734920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3802f-87f6-486a-aabf-5a8d1d2d8d94 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:52:56.735351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3802f-87f6-486a-aabf-5a8d1d2d8d94 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:52:56.735494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3802f-87f6-486a-aabf-5a8d1d2d8d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:52:56.793313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0bd3802f-87f6-486a-aabf-5a8d1d2d8d94 None None] MFA Rules not processed for user `97524081e59f4775b83d82edf737810e`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:52:56.794577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0bd3802f-87f6-486a-aabf-5a8d1d2d8d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:52:56.811444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:52:56.865906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 762/1518] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Jul 14 15:52:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:53:00.979817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947dce6-ab6a-4492-91c6-3c02e06078e8 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:00.980325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947dce6-ab6a-4492-91c6-3c02e06078e8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:00.980440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947dce6-ab6a-4492-91c6-3c02e06078e8 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:00.981746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 757/1519] 10.222.0.47 () {54 vars in 960 bytes} [Thu Jul 14 15:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:53:00.986917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330ea17-ed7c-4771-9c54-81f21e056b11 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:00.987328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330ea17-ed7c-4771-9c54-81f21e056b11 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:00.987506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330ea17-ed7c-4771-9c54-81f21e056b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:01.005612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e330ea17-ed7c-4771-9c54-81f21e056b11 None None] MFA Rules not processed for user `97524081e59f4775b83d82edf737810e`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:53:01.006838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e330ea17-ed7c-4771-9c54-81f21e056b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:01.015579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 763/1520] 10.222.0.47 () {58 vars in 1057 bytes} [Thu Jul 14 15:53:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:53:01.023607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947dce6-ab6a-4492-91c6-3c02e06078e8 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:53:01.038551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f947dce6-ab6a-4492-91c6-3c02e06078e8 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:53:01.044332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0f54a-19b7-4a49-89ac-3f8dce9b93bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:53:01.044989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0f54a-19b7-4a49-89ac-3f8dce9b93bd service placement] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:01.045301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0f54a-19b7-4a49-89ac-3f8dce9b93bd service placement] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:01.045578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0f54a-19b7-4a49-89ac-3f8dce9b93bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:01.055383 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0f54a-19b7-4a49-89ac-3f8dce9b93bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:53:01.056974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0f54a-19b7-4a49-89ac-3f8dce9b93bd service placement] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:53:01.067923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 758/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:53:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:53:06.594152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cc3ec-a9d7-4f41-9a20-44dc889973a7 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:06.594688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cc3ec-a9d7-4f41-9a20-44dc889973a7 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:06.594798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cc3ec-a9d7-4f41-9a20-44dc889973a7 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:06.596686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:53:06.601563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87509b65-dcb3-484e-a3cd-10b8ead36f0e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:06.602037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87509b65-dcb3-484e-a3cd-10b8ead36f0e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:06.602144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87509b65-dcb3-484e-a3cd-10b8ead36f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:06.620031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-87509b65-dcb3-484e-a3cd-10b8ead36f0e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:53:06.621238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-87509b65-dcb3-484e-a3cd-10b8ead36f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:06.629612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:53:06.783975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cc3ec-a9d7-4f41-9a20-44dc889973a7 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:53:06.798291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cc3ec-a9d7-4f41-9a20-44dc889973a7 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:53:06.804009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc57ce-fec4-4453-bc6b-fd16ede9dcbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:53:06.804684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc57ce-fec4-4453-bc6b-fd16ede9dcbb service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:06.804977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc57ce-fec4-4453-bc6b-fd16ede9dcbb service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:06.805257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc57ce-fec4-4453-bc6b-fd16ede9dcbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:06.814225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc57ce-fec4-4453-bc6b-fd16ede9dcbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:53:06.815807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc57ce-fec4-4453-bc6b-fd16ede9dcbb service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:53:06.825366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 765/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:53:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:53:07.651275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8969f2a-b553-4b75-972f-89b77428b9ba None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:07.651917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8969f2a-b553-4b75-972f-89b77428b9ba None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:07.651917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8969f2a-b553-4b75-972f-89b77428b9ba None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:07.653351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 15:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:53:07.659187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadb640-03dd-4dc2-8665-bdbe4efdcbf6 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:07.659636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadb640-03dd-4dc2-8665-bdbe4efdcbf6 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:07.659739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadb640-03dd-4dc2-8665-bdbe4efdcbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:07.678554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-fdadb640-03dd-4dc2-8665-bdbe4efdcbf6 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:53:07.679813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fdadb640-03dd-4dc2-8665-bdbe4efdcbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:07.687598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 15:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:53:07.838565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8969f2a-b553-4b75-972f-89b77428b9ba None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:53:07.853412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8969f2a-b553-4b75-972f-89b77428b9ba None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:53:07.858943 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb8ddc-5f8e-4186-a83b-ac1f72ded009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:53:07.859583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb8ddc-5f8e-4186-a83b-ac1f72ded009 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:07.859868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb8ddc-5f8e-4186-a83b-ac1f72ded009 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:07.860148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb8ddc-5f8e-4186-a83b-ac1f72ded009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:07.868232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb8ddc-5f8e-4186-a83b-ac1f72ded009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:53:07.869877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb8ddc-5f8e-4186-a83b-ac1f72ded009 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:53:07.879312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 761/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:53:11.665472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f333a3a-2fc1-459e-8911-e70857b4ab8a None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:11.666034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f333a3a-2fc1-459e-8911-e70857b4ab8a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:11.666034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f333a3a-2fc1-459e-8911-e70857b4ab8a None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:11.667471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:53:11.672653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c5828-06c1-414f-86f1-54cebc1474fe None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:11.673068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c5828-06c1-414f-86f1-54cebc1474fe None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:11.673208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c5828-06c1-414f-86f1-54cebc1474fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:11.692426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-764c5828-06c1-414f-86f1-54cebc1474fe None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:53:11.693563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-764c5828-06c1-414f-86f1-54cebc1474fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:11.701470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:53:11.772893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f333a3a-2fc1-459e-8911-e70857b4ab8a None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:53:11.774649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4f333a3a-2fc1-459e-8911-e70857b4ab8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:11.853527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f333a3a-2fc1-459e-8911-e70857b4ab8a None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:53:11.857098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaacef-93cb-4e0d-8566-cd58c96b3c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:53:11.857744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaacef-93cb-4e0d-8566-cd58c96b3c07 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:11.858056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaacef-93cb-4e0d-8566-cd58c96b3c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:11.858334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaacef-93cb-4e0d-8566-cd58c96b3c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:11.867849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaacef-93cb-4e0d-8566-cd58c96b3c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:53:11.869282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaacef-93cb-4e0d-8566-cd58c96b3c07 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:53:11.879201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 768/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:53:13.901129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f5f28-f695-43eb-8217-cabee5a28e4e None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:13.901731 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f5f28-f695-43eb-8217-cabee5a28e4e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:13.901731 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f5f28-f695-43eb-8217-cabee5a28e4e None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:13.903089 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:53:13.908148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accdf3c-e1fe-47af-a745-22511686f828 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:13.908593 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accdf3c-e1fe-47af-a745-22511686f828 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:13.908765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accdf3c-e1fe-47af-a745-22511686f828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:13.926633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8accdf3c-e1fe-47af-a745-22511686f828 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:53:13.928082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8accdf3c-e1fe-47af-a745-22511686f828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:13.935852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:53:13.971522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f5f28-f695-43eb-8217-cabee5a28e4e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:53:13.982691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f5f28-f695-43eb-8217-cabee5a28e4e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:53:13.986188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081f39f-0788-40bd-86fc-972534f39477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:53:13.986857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f39f-0788-40bd-86fc-972534f39477 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:13.987138 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f39f-0788-40bd-86fc-972534f39477 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:13.987433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f39f-0788-40bd-86fc-972534f39477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:13.996213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081f39f-0788-40bd-86fc-972534f39477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:53:13.997642 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081f39f-0788-40bd-86fc-972534f39477 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:53:14.006979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 764/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:53:17.479476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22322c3d-9e6f-4175-bb8f-79ad8711ade3 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:17.479940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22322c3d-9e6f-4175-bb8f-79ad8711ade3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:17.480045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22322c3d-9e6f-4175-bb8f-79ad8711ade3 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:17.481393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:53:17.486483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3afc5-a6e1-4f6f-abc4-2405b58c0ff7 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:17.486950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3afc5-a6e1-4f6f-abc4-2405b58c0ff7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:17.487056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3afc5-a6e1-4f6f-abc4-2405b58c0ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:17.505251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-13b3afc5-a6e1-4f6f-abc4-2405b58c0ff7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:53:17.506488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-13b3afc5-a6e1-4f6f-abc4-2405b58c0ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:17.514293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:53:17.815706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22322c3d-9e6f-4175-bb8f-79ad8711ade3 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:53:17.817372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-22322c3d-9e6f-4175-bb8f-79ad8711ade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:17.882636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:53:17.895711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22322c3d-9e6f-4175-bb8f-79ad8711ade3 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:53:17.899273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e39af2-9eed-4e68-933b-732f9e16af22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:53:17.899920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e39af2-9eed-4e68-933b-732f9e16af22 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:17.900196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e39af2-9eed-4e68-933b-732f9e16af22 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:17.900469 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e39af2-9eed-4e68-933b-732f9e16af22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:17.909896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e39af2-9eed-4e68-933b-732f9e16af22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:53:17.911675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e39af2-9eed-4e68-933b-732f9e16af22 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:53:17.921294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 771/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:53:19.955134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa39809-53af-4684-9eb4-46599f475379 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:19.955708 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa39809-53af-4684-9eb4-46599f475379 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:19.955821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa39809-53af-4684-9eb4-46599f475379 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:19.957430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 15:53:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:53:19.962572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44a559-0740-4323-99bb-0cd2f7ba20b8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:19.963020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44a559-0740-4323-99bb-0cd2f7ba20b8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:19.963126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44a559-0740-4323-99bb-0cd2f7ba20b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:19.980399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-3b44a559-0740-4323-99bb-0cd2f7ba20b8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:53:19.981612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3b44a559-0740-4323-99bb-0cd2f7ba20b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:19.989987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 15:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:53:20.289132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa39809-53af-4684-9eb4-46599f475379 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:53:20.290911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3aa39809-53af-4684-9eb4-46599f475379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:53:20.298520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=84 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:53:20.305083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa39809-53af-4684-9eb4-46599f475379 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:53:20.323718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc6767-9c35-4ddf-b57b-9fbfe8997223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:53:20.323718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6767-9c35-4ddf-b57b-9fbfe8997223 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:53:20.323718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6767-9c35-4ddf-b57b-9fbfe8997223 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:53:20.323718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6767-9c35-4ddf-b57b-9fbfe8997223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:53:20.324194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc6767-9c35-4ddf-b57b-9fbfe8997223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:53:20.324194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc6767-9c35-4ddf-b57b-9fbfe8997223 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:53:20.329647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 767/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:53:29.975687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=11 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:53:30.339569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:50.651355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f1cf9-dbe1-4a63-84ed-e936244ba677 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:50.652099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f1cf9-dbe1-4a63-84ed-e936244ba677 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:50.652099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f1cf9-dbe1-4a63-84ed-e936244ba677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:50.679319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-be5f1cf9-dbe1-4a63-84ed-e936244ba677 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:50.680763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-be5f1cf9-dbe1-4a63-84ed-e936244ba677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:50.689696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 773/1540] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:50.699521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b28208-b7cc-45a0-b57e-2e3b7e687f0c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:50.700545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b28208-b7cc-45a0-b57e-2e3b7e687f0c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:50.701054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b28208-b7cc-45a0-b57e-2e3b7e687f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:50.731147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-18b28208-b7cc-45a0-b57e-2e3b7e687f0c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:50.732858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-18b28208-b7cc-45a0-b57e-2e3b7e687f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:50.748711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 768/1541] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:50.753740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f1cf9-dbe1-4a63-84ed-e936244ba677 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:50.764908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f1cf9-dbe1-4a63-84ed-e936244ba677 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:50.774326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd157e34-f74d-4bec-83a0-d21b91e4fe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:50.775009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd157e34-f74d-4bec-83a0-d21b91e4fe24 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:50.775296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd157e34-f74d-4bec-83a0-d21b91e4fe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:50.775577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd157e34-f74d-4bec-83a0-d21b91e4fe24 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:50.776312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd157e34-f74d-4bec-83a0-d21b91e4fe24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:54:50.776622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd157e34-f74d-4bec-83a0-d21b91e4fe24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:50.778251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd157e34-f74d-4bec-83a0-d21b91e4fe24 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:50.796367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 774/1542] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:54:50.805592 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b28208-b7cc-45a0-b57e-2e3b7e687f0c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:50.820589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b28208-b7cc-45a0-b57e-2e3b7e687f0c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:50.835806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920b58a-3840-48d7-8ac0-6f1a707e8004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:50.837075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920b58a-3840-48d7-8ac0-6f1a707e8004 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:50.837593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920b58a-3840-48d7-8ac0-6f1a707e8004 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:50.838202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920b58a-3840-48d7-8ac0-6f1a707e8004 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:50.839228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920b58a-3840-48d7-8ac0-6f1a707e8004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:50.841438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920b58a-3840-48d7-8ac0-6f1a707e8004 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:50.993851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 769/1543] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:50 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:50.997897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd157e34-f74d-4bec-83a0-d21b91e4fe24 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.015864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd157e34-f74d-4bec-83a0-d21b91e4fe24 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.113847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a1ecb-1428-4181-a131-afa0eb96ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.113847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a1ecb-1428-4181-a131-afa0eb96ec08 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.113847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a1ecb-1428-4181-a131-afa0eb96ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.113847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a1ecb-1428-4181-a131-afa0eb96ec08 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.114412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a1ecb-1428-4181-a131-afa0eb96ec08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.118829 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a1ecb-1428-4181-a131-afa0eb96ec08 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.123291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8f361-a0a2-49c9-bec8-47efd8341aeb None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.123688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8f361-a0a2-49c9-bec8-47efd8341aeb None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.123977 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8f361-a0a2-49c9-bec8-47efd8341aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.163382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a5c8f361-a0a2-49c9-bec8-47efd8341aeb None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.165345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a5c8f361-a0a2-49c9-bec8-47efd8341aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.180072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 770/1544] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.181499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a1431-3ce1-4dfa-b253-497a20cdd3eb None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.181799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a1431-3ce1-4dfa-b253-497a20cdd3eb None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.182133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a1431-3ce1-4dfa-b253-497a20cdd3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.210632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-516a1431-3ce1-4dfa-b253-497a20cdd3eb None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.211857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-516a1431-3ce1-4dfa-b253-497a20cdd3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.223976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 775/1545] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:50 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:54:51.228666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68835fa-d433-4dc2-831e-c297ccc5b057 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.228967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68835fa-d433-4dc2-831e-c297ccc5b057 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.229243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68835fa-d433-4dc2-831e-c297ccc5b057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.232087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 771/1546] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.232804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a1431-3ce1-4dfa-b253-497a20cdd3eb None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.255961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a1431-3ce1-4dfa-b253-497a20cdd3eb None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.270731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a68835fa-d433-4dc2-831e-c297ccc5b057 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.271604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fdf76-f8b8-4af3-84b9-151337618957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.272125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a68835fa-d433-4dc2-831e-c297ccc5b057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.272462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fdf76-f8b8-4af3-84b9-151337618957 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.272762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fdf76-f8b8-4af3-84b9-151337618957 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.273070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fdf76-f8b8-4af3-84b9-151337618957 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.273810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fdf76-f8b8-4af3-84b9-151337618957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.275442 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fdf76-f8b8-4af3-84b9-151337618957 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.294190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 776/1547] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.294872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 772/1548] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:51.295541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e595c8-94b1-48ad-9424-b89adb768bcc None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.298374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e595c8-94b1-48ad-9424-b89adb768bcc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.298374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e595c8-94b1-48ad-9424-b89adb768bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.299268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fdf76-f8b8-4af3-84b9-151337618957 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.317388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fdf76-f8b8-4af3-84b9-151337618957 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.331052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfaf0f-5539-43bd-b77e-f85457eee9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.331774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfaf0f-5539-43bd-b77e-f85457eee9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.332047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfaf0f-5539-43bd-b77e-f85457eee9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.332332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfaf0f-5539-43bd-b77e-f85457eee9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.333074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dfaf0f-5539-43bd-b77e-f85457eee9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.333589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d2e595c8-94b1-48ad-9424-b89adb768bcc None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.334736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dfaf0f-5539-43bd-b77e-f85457eee9d3 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.335281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d2e595c8-94b1-48ad-9424-b89adb768bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.345593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 773/1549] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:51.346459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfaf0f-5539-43bd-b77e-f85457eee9d3 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.351974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 777/1550] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.353599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996281c-ec28-427e-944a-6fcbdff6f581 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.354219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996281c-ec28-427e-944a-6fcbdff6f581 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.354704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996281c-ec28-427e-944a-6fcbdff6f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.359940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfaf0f-5539-43bd-b77e-f85457eee9d3 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.376871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26837054-edb0-4350-b968-e76fc638df16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.376871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26837054-edb0-4350-b968-e76fc638df16 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.376871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26837054-edb0-4350-b968-e76fc638df16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.376871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26837054-edb0-4350-b968-e76fc638df16 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.377320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26837054-edb0-4350-b968-e76fc638df16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:54:51.377577 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26837054-edb0-4350-b968-e76fc638df16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.378520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WARNING py.warnings [None req-26837054-edb0-4350-b968-e76fc638df16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:54:51.378520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: warnings.warn(msg) Jul 14 15:54:51.378520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: Jul 14 15:54:51.379416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26837054-edb0-4350-b968-e76fc638df16 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.384854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-3996281c-ec28-427e-944a-6fcbdff6f581 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.386341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3996281c-ec28-427e-944a-6fcbdff6f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.403161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 774/1551] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:54:51.404036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 778/1552] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.404717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30699ebb-38bf-4b5d-842a-ebe35e7cd66e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.404859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996281c-ec28-427e-944a-6fcbdff6f581 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.405042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30699ebb-38bf-4b5d-842a-ebe35e7cd66e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.405304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30699ebb-38bf-4b5d-842a-ebe35e7cd66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.420682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996281c-ec28-427e-944a-6fcbdff6f581 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.425649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1039680-f028-441e-a688-8e443594e62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.427353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1039680-f028-441e-a688-8e443594e62b admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.427873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1039680-f028-441e-a688-8e443594e62b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.428390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1039680-f028-441e-a688-8e443594e62b admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.429959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1039680-f028-441e-a688-8e443594e62b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:54:51.430478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1039680-f028-441e-a688-8e443594e62b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.432439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1039680-f028-441e-a688-8e443594e62b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.435991 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-30699ebb-38bf-4b5d-842a-ebe35e7cd66e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.437220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-30699ebb-38bf-4b5d-842a-ebe35e7cd66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.446039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 779/1553] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:54:51.446292 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 775/1554] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.447067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30699ebb-38bf-4b5d-842a-ebe35e7cd66e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.454875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aebf0-f1af-4a30-b98c-f96be05adef9 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.457255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aebf0-f1af-4a30-b98c-f96be05adef9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.458779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aebf0-f1af-4a30-b98c-f96be05adef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.468091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30699ebb-38bf-4b5d-842a-ebe35e7cd66e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.481275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad62f6-c3fd-44ed-9d86-3dff7f923357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.482227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad62f6-c3fd-44ed-9d86-3dff7f923357 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.482534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad62f6-c3fd-44ed-9d86-3dff7f923357 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.482810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad62f6-c3fd-44ed-9d86-3dff7f923357 admin admin] PATH_INFO: `/v3/projects/ce8ab327172f416ea982936b287762d2/users/876156edcf9643beaf74738d31177041/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.484081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-728aebf0-f1af-4a30-b98c-f96be05adef9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.485497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-728aebf0-f1af-4a30-b98c-f96be05adef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.495438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 780/1555] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.496360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728aebf0-f1af-4a30-b98c-f96be05adef9 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.498803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad62f6-c3fd-44ed-9d86-3dff7f923357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8ab327172f416ea982936b287762d2, user_id=876156edcf9643beaf74738d31177041, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.501584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad62f6-c3fd-44ed-9d86-3dff7f923357 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.506136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728aebf0-f1af-4a30-b98c-f96be05adef9 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.510864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534883da-ba89-4ae2-929b-7adb639b780a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.518397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534883da-ba89-4ae2-929b-7adb639b780a admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.518397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534883da-ba89-4ae2-929b-7adb639b780a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.518397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534883da-ba89-4ae2-929b-7adb639b780a admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.518397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534883da-ba89-4ae2-929b-7adb639b780a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.518397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING py.warnings [None req-534883da-ba89-4ae2-929b-7adb639b780a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:54:51.518397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: warnings.warn(msg) Jul 14 15:54:51.518397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: Jul 14 15:54:51.519025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534883da-ba89-4ae2-929b-7adb639b780a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.607674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 776/1556] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:51 2022] PUT /identity/v3/projects/ce8ab327172f416ea982936b287762d2/users/876156edcf9643beaf74738d31177041/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:51.608896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19377266-5429-47d4-9287-bbc32228cf76 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.609182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19377266-5429-47d4-9287-bbc32228cf76 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.609454 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19377266-5429-47d4-9287-bbc32228cf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.623755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 781/1557] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:51.623755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8920d-5899-44a4-9a97-d121e4949085 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.623755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8920d-5899-44a4-9a97-d121e4949085 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.623755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8920d-5899-44a4-9a97-d121e4949085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.630116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-19377266-5429-47d4-9287-bbc32228cf76 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.631342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-19377266-5429-47d4-9287-bbc32228cf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.641170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-23a8920d-5899-44a4-9a97-d121e4949085 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.642384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-23a8920d-5899-44a4-9a97-d121e4949085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.690536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 777/1558] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.691368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19377266-5429-47d4-9287-bbc32228cf76 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.700372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19377266-5429-47d4-9287-bbc32228cf76 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.703767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 782/1559] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.705150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc043e6-3402-4cec-9f84-c88630d5472a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.705472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc043e6-3402-4cec-9f84-c88630d5472a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.705695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570db7b-b97a-4034-a13d-d9eb6ccb3b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.705863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc043e6-3402-4cec-9f84-c88630d5472a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.706501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570db7b-b97a-4034-a13d-d9eb6ccb3b32 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.706859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570db7b-b97a-4034-a13d-d9eb6ccb3b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.707372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570db7b-b97a-4034-a13d-d9eb6ccb3b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.707974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570db7b-b97a-4034-a13d-d9eb6ccb3b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.710039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570db7b-b97a-4034-a13d-d9eb6ccb3b32 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.725293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-4bc043e6-3402-4cec-9f84-c88630d5472a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.727284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4bc043e6-3402-4cec-9f84-c88630d5472a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.735110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 783/1560] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.737392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452921f9-2f90-4d96-9a58-55f0100c7d0a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.737392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452921f9-2f90-4d96-9a58-55f0100c7d0a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.737392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452921f9-2f90-4d96-9a58-55f0100c7d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.742931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:51.755019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-452921f9-2f90-4d96-9a58-55f0100c7d0a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.756223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-452921f9-2f90-4d96-9a58-55f0100c7d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.764184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 784/1561] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.766101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84186c1a-5f8f-4313-a0ab-7c414cd490c0 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.766101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84186c1a-5f8f-4313-a0ab-7c414cd490c0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.766101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84186c1a-5f8f-4313-a0ab-7c414cd490c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.784249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-84186c1a-5f8f-4313-a0ab-7c414cd490c0 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.785422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-84186c1a-5f8f-4313-a0ab-7c414cd490c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.792863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 778/1562] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:51.793000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49844c3-3cd2-4e04-af6d-b1bf4d80f4c0 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.800723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49844c3-3cd2-4e04-af6d-b1bf4d80f4c0 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.800723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49844c3-3cd2-4e04-af6d-b1bf4d80f4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.800938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 785/1563] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.800938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84186c1a-5f8f-4313-a0ab-7c414cd490c0 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.807982 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84186c1a-5f8f-4313-a0ab-7c414cd490c0 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.821160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-b49844c3-3cd2-4e04-af6d-b1bf4d80f4c0 None None] MFA Rules not processed for user `876156edcf9643beaf74738d31177041`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.822546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b49844c3-3cd2-4e04-af6d-b1bf4d80f4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.831918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:51.864857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca96b40-0e07-4ebe-b2d6-158d76f3dbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.865463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca96b40-0e07-4ebe-b2d6-158d76f3dbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.865745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca96b40-0e07-4ebe-b2d6-158d76f3dbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.866056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca96b40-0e07-4ebe-b2d6-158d76f3dbd8 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.866748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca96b40-0e07-4ebe-b2d6-158d76f3dbd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.868708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca96b40-0e07-4ebe-b2d6-158d76f3dbd8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.887898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 779/1564] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.888838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49844c3-3cd2-4e04-af6d-b1bf4d80f4c0 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.899700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49844c3-3cd2-4e04-af6d-b1bf4d80f4c0 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.904344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba515d-4758-4b3a-ad96-c4b4a82d3ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.904992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba515d-4758-4b3a-ad96-c4b4a82d3ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.905275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba515d-4758-4b3a-ad96-c4b4a82d3ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.905548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba515d-4758-4b3a-ad96-c4b4a82d3ed7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.906310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba515d-4758-4b3a-ad96-c4b4a82d3ed7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:54:51.906613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba515d-4758-4b3a-ad96-c4b4a82d3ed7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.908299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba515d-4758-4b3a-ad96-c4b4a82d3ed7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.924846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 780/1565] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:54:51.924846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba515d-4758-4b3a-ad96-c4b4a82d3ed7 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:51.929989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba515d-4758-4b3a-ad96-c4b4a82d3ed7 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:51.934647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726729b-c366-4ea3-a7d7-21589b3c1569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:51.935273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726729b-c366-4ea3-a7d7-21589b3c1569 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.935560 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726729b-c366-4ea3-a7d7-21589b3c1569 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.935837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726729b-c366-4ea3-a7d7-21589b3c1569 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.936540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726729b-c366-4ea3-a7d7-21589b3c1569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:54:51.936878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726729b-c366-4ea3-a7d7-21589b3c1569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:51.938482 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726729b-c366-4ea3-a7d7-21589b3c1569 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:51.950735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 781/1566] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:54:51.952025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6a7e8-5c57-43e2-93a6-090bba0adb84 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.952369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6a7e8-5c57-43e2-93a6-090bba0adb84 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.952637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6a7e8-5c57-43e2-93a6-090bba0adb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.970602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-74a6a7e8-5c57-43e2-93a6-090bba0adb84 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:51.971853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-74a6a7e8-5c57-43e2-93a6-090bba0adb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:51.980036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 782/1567] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:51.981212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa5671-68a6-4758-856d-2fbc95f8a249 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:51.981504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa5671-68a6-4758-856d-2fbc95f8a249 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:51.982177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa5671-68a6-4758-856d-2fbc95f8a249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:51.996022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 786/1568] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.001318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca96b40-0e07-4ebe-b2d6-158d76f3dbd8 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.001440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e9aa5671-68a6-4758-856d-2fbc95f8a249 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:52.001644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e9aa5671-68a6-4758-856d-2fbc95f8a249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:52.006812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca96b40-0e07-4ebe-b2d6-158d76f3dbd8 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.014050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 783/1569] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:52.014050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa5671-68a6-4758-856d-2fbc95f8a249 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.014296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa4fa1-09d9-4547-800f-274e3861c2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.014296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa4fa1-09d9-4547-800f-274e3861c2dd admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.014296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa4fa1-09d9-4547-800f-274e3861c2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.014296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa4fa1-09d9-4547-800f-274e3861c2dd admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.014662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa4fa1-09d9-4547-800f-274e3861c2dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.015999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aa4fa1-09d9-4547-800f-274e3861c2dd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.025043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa5671-68a6-4758-856d-2fbc95f8a249 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.029871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d2266-748f-46c4-8557-977c3ed66548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.030543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d2266-748f-46c4-8557-977c3ed66548 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.030831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d2266-748f-46c4-8557-977c3ed66548 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.031113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d2266-748f-46c4-8557-977c3ed66548 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.031850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d2266-748f-46c4-8557-977c3ed66548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:54:52.032160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d2266-748f-46c4-8557-977c3ed66548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.033780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d2266-748f-46c4-8557-977c3ed66548 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.046367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 784/1570] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:54:52.047349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d2266-748f-46c4-8557-977c3ed66548 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.057253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d2266-748f-46c4-8557-977c3ed66548 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.061720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb9e45-37fc-4aea-8f0c-40496313f643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.062389 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb9e45-37fc-4aea-8f0c-40496313f643 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.062682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb9e45-37fc-4aea-8f0c-40496313f643 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.062926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb9e45-37fc-4aea-8f0c-40496313f643 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.063594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb9e45-37fc-4aea-8f0c-40496313f643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.065483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb9e45-37fc-4aea-8f0c-40496313f643 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.093727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 787/1571] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:51 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.094639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa4fa1-09d9-4547-800f-274e3861c2dd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.104455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aa4fa1-09d9-4547-800f-274e3861c2dd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.109210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23775c8-48f5-4e47-b2e4-abb115602e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.109896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23775c8-48f5-4e47-b2e4-abb115602e71 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.110182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23775c8-48f5-4e47-b2e4-abb115602e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.110466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23775c8-48f5-4e47-b2e4-abb115602e71 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.111146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23775c8-48f5-4e47-b2e4-abb115602e71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.113051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23775c8-48f5-4e47-b2e4-abb115602e71 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.151674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 785/1572] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.152430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb9e45-37fc-4aea-8f0c-40496313f643 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.213656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb9e45-37fc-4aea-8f0c-40496313f643 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.218612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f481d94-e32e-46c9-a591-cb17cb2a845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.219539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f481d94-e32e-46c9-a591-cb17cb2a845e admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.219539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f481d94-e32e-46c9-a591-cb17cb2a845e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.219831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f481d94-e32e-46c9-a591-cb17cb2a845e admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.220527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f481d94-e32e-46c9-a591-cb17cb2a845e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.222654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f481d94-e32e-46c9-a591-cb17cb2a845e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.229805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 788/1573] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.229805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23775c8-48f5-4e47-b2e4-abb115602e71 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.289334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23775c8-48f5-4e47-b2e4-abb115602e71 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.294070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b06e0-bdc1-4e39-9c21-8b5cf6a07dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.294731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b06e0-bdc1-4e39-9c21-8b5cf6a07dce admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.295014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b06e0-bdc1-4e39-9c21-8b5cf6a07dce admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.295291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b06e0-bdc1-4e39-9c21-8b5cf6a07dce admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.296011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b06e0-bdc1-4e39-9c21-8b5cf6a07dce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:54:52.296315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b06e0-bdc1-4e39-9c21-8b5cf6a07dce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.298059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b06e0-bdc1-4e39-9c21-8b5cf6a07dce admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.306764 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 786/1574] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.307610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f481d94-e32e-46c9-a591-cb17cb2a845e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.311174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 789/1575] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:54:52.312083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b06e0-bdc1-4e39-9c21-8b5cf6a07dce admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.328112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b06e0-bdc1-4e39-9c21-8b5cf6a07dce admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.334065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:52.358143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:52.373225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f481d94-e32e-46c9-a591-cb17cb2a845e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.377843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23e2e2-c815-4604-86b3-2246c70adaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.378528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e2e2-c815-4604-86b3-2246c70adaec admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.378801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e2e2-c815-4604-86b3-2246c70adaec admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.379080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e2e2-c815-4604-86b3-2246c70adaec admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.379812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23e2e2-c815-4604-86b3-2246c70adaec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.381418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23e2e2-c815-4604-86b3-2246c70adaec admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.384939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56e99e-f269-4c69-a153-f8cdcc54bfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.393029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56e99e-f269-4c69-a153-f8cdcc54bfa2 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.393029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56e99e-f269-4c69-a153-f8cdcc54bfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.393029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56e99e-f269-4c69-a153-f8cdcc54bfa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.393029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56e99e-f269-4c69-a153-f8cdcc54bfa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:54:52.393029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56e99e-f269-4c69-a153-f8cdcc54bfa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.393029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56e99e-f269-4c69-a153-f8cdcc54bfa2 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.393618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 787/1576] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:52.393618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23e2e2-c815-4604-86b3-2246c70adaec admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.404475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23e2e2-c815-4604-86b3-2246c70adaec admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.412496 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 790/1577] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:54:52.412496 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56e99e-f269-4c69-a153-f8cdcc54bfa2 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.416576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56e99e-f269-4c69-a153-f8cdcc54bfa2 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.416712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c81985-c0e0-4158-8e69-4daded0f888f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.417430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c81985-c0e0-4158-8e69-4daded0f888f admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.417778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c81985-c0e0-4158-8e69-4daded0f888f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.418148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c81985-c0e0-4158-8e69-4daded0f888f admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.418918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c81985-c0e0-4158-8e69-4daded0f888f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.420882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c81985-c0e0-4158-8e69-4daded0f888f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.421304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10520098-68fb-4fac-b390-af2b0f49baaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.422077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10520098-68fb-4fac-b390-af2b0f49baaf admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.422366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10520098-68fb-4fac-b390-af2b0f49baaf admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.422649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10520098-68fb-4fac-b390-af2b0f49baaf admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.423374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10520098-68fb-4fac-b390-af2b0f49baaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.424957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10520098-68fb-4fac-b390-af2b0f49baaf admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.435874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 791/1578] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:52.436993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10520098-68fb-4fac-b390-af2b0f49baaf admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.447035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10520098-68fb-4fac-b390-af2b0f49baaf admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.451583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f737b2-05cc-48f0-a132-5add81b1d78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.452242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f737b2-05cc-48f0-a132-5add81b1d78d admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.452505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f737b2-05cc-48f0-a132-5add81b1d78d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.452783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f737b2-05cc-48f0-a132-5add81b1d78d admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.453461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f737b2-05cc-48f0-a132-5add81b1d78d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.455392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f737b2-05cc-48f0-a132-5add81b1d78d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.540887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 788/1579] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/projects => generated 401 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.541716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c81985-c0e0-4158-8e69-4daded0f888f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.599224 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 792/1580] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.599224 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f737b2-05cc-48f0-a132-5add81b1d78d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.605417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c81985-c0e0-4158-8e69-4daded0f888f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.606468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f737b2-05cc-48f0-a132-5add81b1d78d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.610242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff988409-58bf-4597-a90c-4860963140b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.610915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff988409-58bf-4597-a90c-4860963140b2 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.611037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996d910-42a6-4ac4-b85c-8d2d4166d662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.611243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff988409-58bf-4597-a90c-4860963140b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.611527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff988409-58bf-4597-a90c-4860963140b2 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.611681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996d910-42a6-4ac4-b85c-8d2d4166d662 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.611930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996d910-42a6-4ac4-b85c-8d2d4166d662 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.612246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996d910-42a6-4ac4-b85c-8d2d4166d662 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.612407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff988409-58bf-4597-a90c-4860963140b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.613361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996d910-42a6-4ac4-b85c-8d2d4166d662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.613488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WARNING py.warnings [None req-ff988409-58bf-4597-a90c-4860963140b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:54:52.613488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: warnings.warn(msg) Jul 14 15:54:52.613488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: Jul 14 15:54:52.614582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff988409-58bf-4597-a90c-4860963140b2 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.614798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996d910-42a6-4ac4-b85c-8d2d4166d662 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.707798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 789/1581] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.708697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff988409-58bf-4597-a90c-4860963140b2 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.718046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff988409-58bf-4597-a90c-4860963140b2 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.721299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 793/1582] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.721299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996d910-42a6-4ac4-b85c-8d2d4166d662 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.723076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcb398-e2e9-49b9-a73d-7e2da9d675a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.723788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcb398-e2e9-49b9-a73d-7e2da9d675a7 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.724104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcb398-e2e9-49b9-a73d-7e2da9d675a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.724587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcb398-e2e9-49b9-a73d-7e2da9d675a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.725285 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcb398-e2e9-49b9-a73d-7e2da9d675a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.728314 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcb398-e2e9-49b9-a73d-7e2da9d675a7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.730192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996d910-42a6-4ac4-b85c-8d2d4166d662 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.734889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed8f02-2311-4880-a5ae-6cfec25b1223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.735549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed8f02-2311-4880-a5ae-6cfec25b1223 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.735838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed8f02-2311-4880-a5ae-6cfec25b1223 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.736119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed8f02-2311-4880-a5ae-6cfec25b1223 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.736850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed8f02-2311-4880-a5ae-6cfec25b1223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.738462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed8f02-2311-4880-a5ae-6cfec25b1223 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.749887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 794/1583] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:52.751540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed8f02-2311-4880-a5ae-6cfec25b1223 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.760094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed8f02-2311-4880-a5ae-6cfec25b1223 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.764748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21810f55-b938-4ddf-a000-70b83016e915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.765369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21810f55-b938-4ddf-a000-70b83016e915 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.765655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21810f55-b938-4ddf-a000-70b83016e915 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.765967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21810f55-b938-4ddf-a000-70b83016e915 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.767685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21810f55-b938-4ddf-a000-70b83016e915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.770033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21810f55-b938-4ddf-a000-70b83016e915 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.807341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 790/1584] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.808320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcb398-e2e9-49b9-a73d-7e2da9d675a7 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.868040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcb398-e2e9-49b9-a73d-7e2da9d675a7 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.872570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e86338-8962-41af-9d83-75c8e932825d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.873218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e86338-8962-41af-9d83-75c8e932825d admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.873498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e86338-8962-41af-9d83-75c8e932825d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.873767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e86338-8962-41af-9d83-75c8e932825d admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.877774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e86338-8962-41af-9d83-75c8e932825d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.877774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e86338-8962-41af-9d83-75c8e932825d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.881251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:52.886093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 791/1585] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:52.886983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e86338-8962-41af-9d83-75c8e932825d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.896451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e86338-8962-41af-9d83-75c8e932825d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.901131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249a425-74c2-48d5-921e-f490382112be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:52.901774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249a425-74c2-48d5-921e-f490382112be admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:52.902079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249a425-74c2-48d5-921e-f490382112be admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:52.906513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249a425-74c2-48d5-921e-f490382112be admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:52.906513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249a425-74c2-48d5-921e-f490382112be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:52.906513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249a425-74c2-48d5-921e-f490382112be admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:52.933067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 795/1586] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/projects => generated 371 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.937051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21810f55-b938-4ddf-a000-70b83016e915 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.988128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 792/1587] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:52 2022] POST /identity/v3/users => generated 301 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:52.988973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249a425-74c2-48d5-921e-f490382112be admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:52.995820 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21810f55-b938-4ddf-a000-70b83016e915 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:52.999412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249a425-74c2-48d5-921e-f490382112be admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.000456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d5099-aee3-4a1e-bbb2-8668e4447fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.001131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d5099-aee3-4a1e-bbb2-8668e4447fcf admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.001418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d5099-aee3-4a1e-bbb2-8668e4447fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.001698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d5099-aee3-4a1e-bbb2-8668e4447fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.002464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d5099-aee3-4a1e-bbb2-8668e4447fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.004147 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d5099-aee3-4a1e-bbb2-8668e4447fcf admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.009616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f933a-f941-48e6-bce7-0cd48c046660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.009616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f933a-f941-48e6-bce7-0cd48c046660 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.009616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f933a-f941-48e6-bce7-0cd48c046660 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.009616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f933a-f941-48e6-bce7-0cd48c046660 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.010050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f933a-f941-48e6-bce7-0cd48c046660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.010050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f933a-f941-48e6-bce7-0cd48c046660 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.014520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 796/1588] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.015356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d5099-aee3-4a1e-bbb2-8668e4447fcf admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.026950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 793/1589] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.026950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f933a-f941-48e6-bce7-0cd48c046660 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.027241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d5099-aee3-4a1e-bbb2-8668e4447fcf admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.030513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26e3c4-d215-4622-89f0-c4c157ac155d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.031171 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26e3c4-d215-4622-89f0-c4c157ac155d admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.032084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26e3c4-d215-4622-89f0-c4c157ac155d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.032377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26e3c4-d215-4622-89f0-c4c157ac155d admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.033131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26e3c4-d215-4622-89f0-c4c157ac155d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.034752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f933a-f941-48e6-bce7-0cd48c046660 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.034889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26e3c4-d215-4622-89f0-c4c157ac155d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.040159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdec75-d37b-4005-bc4d-df47115f0126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.044018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdec75-d37b-4005-bc4d-df47115f0126 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.044018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdec75-d37b-4005-bc4d-df47115f0126 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.044018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdec75-d37b-4005-bc4d-df47115f0126 admin admin] PATH_INFO: `/v3/projects/64a86c4d5f1043ca93987ed2c77a3629/users/dcdbbbf1b9944b09898fa8731184c845/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.050233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 797/1590] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.051050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26e3c4-d215-4622-89f0-c4c157ac155d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.060878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdec75-d37b-4005-bc4d-df47115f0126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a86c4d5f1043ca93987ed2c77a3629, user_id=dcdbbbf1b9944b09898fa8731184c845, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.063012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26e3c4-d215-4622-89f0-c4c157ac155d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.063640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdec75-d37b-4005-bc4d-df47115f0126 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.067904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc2e60-98fc-4b28-a7ac-24ac5336645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.068567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc2e60-98fc-4b28-a7ac-24ac5336645d admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.068881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc2e60-98fc-4b28-a7ac-24ac5336645d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.069160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc2e60-98fc-4b28-a7ac-24ac5336645d admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.070507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc2e60-98fc-4b28-a7ac-24ac5336645d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.072420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc2e60-98fc-4b28-a7ac-24ac5336645d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.154000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 794/1591] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:53 2022] PUT /identity/v3/projects/64a86c4d5f1043ca93987ed2c77a3629/users/dcdbbbf1b9944b09898fa8731184c845/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:53.154806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdec75-d37b-4005-bc4d-df47115f0126 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.179053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 798/1592] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:53 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:53.179053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc2e60-98fc-4b28-a7ac-24ac5336645d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.188687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc2e60-98fc-4b28-a7ac-24ac5336645d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.201077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:53.218602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdec75-d37b-4005-bc4d-df47115f0126 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.223223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44c6ee-acb6-4d09-b163-c3e8e629629d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.224118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c6ee-acb6-4d09-b163-c3e8e629629d admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.224401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c6ee-acb6-4d09-b163-c3e8e629629d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.224802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44c6ee-acb6-4d09-b163-c3e8e629629d admin admin] PATH_INFO: `/v3/projects/73dcc59c5829442da0518976525fe40b/users/d476a2bec8164b8bbf9a5aaa419a0351/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.239277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44c6ee-acb6-4d09-b163-c3e8e629629d admin admin] RBAC: Authorizing `identity:create_grant(project_id=73dcc59c5829442da0518976525fe40b, user_id=d476a2bec8164b8bbf9a5aaa419a0351, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.242417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44c6ee-acb6-4d09-b163-c3e8e629629d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.246121 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fd9f8-712a-4010-a1da-03155f01bab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.247074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fd9f8-712a-4010-a1da-03155f01bab6 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.247800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fd9f8-712a-4010-a1da-03155f01bab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.248133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fd9f8-712a-4010-a1da-03155f01bab6 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.248855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fd9f8-712a-4010-a1da-03155f01bab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.250810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fd9f8-712a-4010-a1da-03155f01bab6 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.347972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 795/1593] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:53 2022] PUT /identity/v3/projects/73dcc59c5829442da0518976525fe40b/users/d476a2bec8164b8bbf9a5aaa419a0351/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:53.348859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44c6ee-acb6-4d09-b163-c3e8e629629d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.378503 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 799/1594] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:53 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:53.378503 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fd9f8-712a-4010-a1da-03155f01bab6 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.388729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fd9f8-712a-4010-a1da-03155f01bab6 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.414393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44c6ee-acb6-4d09-b163-c3e8e629629d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.419122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74a5e9-2e8b-4858-b1ea-8e2550639dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.419794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74a5e9-2e8b-4858-b1ea-8e2550639dda admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.420077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74a5e9-2e8b-4858-b1ea-8e2550639dda admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.420349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74a5e9-2e8b-4858-b1ea-8e2550639dda admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.421066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74a5e9-2e8b-4858-b1ea-8e2550639dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.422687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74a5e9-2e8b-4858-b1ea-8e2550639dda admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.433504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 796/1595] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.434339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74a5e9-2e8b-4858-b1ea-8e2550639dda admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.443505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74a5e9-2e8b-4858-b1ea-8e2550639dda admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.448656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed991a8-0264-4f9d-a3ca-b52c1be9c2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.449267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed991a8-0264-4f9d-a3ca-b52c1be9c2de admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.449549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed991a8-0264-4f9d-a3ca-b52c1be9c2de admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.449840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed991a8-0264-4f9d-a3ca-b52c1be9c2de admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.450544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed991a8-0264-4f9d-a3ca-b52c1be9c2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.452096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed991a8-0264-4f9d-a3ca-b52c1be9c2de admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.452210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaad414-889d-4621-87b6-d8d0a3df4a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.453110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaad414-889d-4621-87b6-d8d0a3df4a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.453395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaad414-889d-4621-87b6-d8d0a3df4a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.453679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaad414-889d-4621-87b6-d8d0a3df4a55 admin admin] PATH_INFO: `/v3/projects/d57676cc6e9c453aad20204f6c6f33de/users/cfc4c09b0a8a4a5db9a4d330250fb2e2/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.462729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 797/1596] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.463657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed991a8-0264-4f9d-a3ca-b52c1be9c2de admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.469993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaad414-889d-4621-87b6-d8d0a3df4a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57676cc6e9c453aad20204f6c6f33de, user_id=cfc4c09b0a8a4a5db9a4d330250fb2e2, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.471026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING py.warnings [None req-ddaad414-889d-4621-87b6-d8d0a3df4a55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:54:53.471026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: warnings.warn(msg) Jul 14 15:54:53.471026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: Jul 14 15:54:53.472920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaad414-889d-4621-87b6-d8d0a3df4a55 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.475900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed991a8-0264-4f9d-a3ca-b52c1be9c2de admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.480501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed69b1e-7dbc-4fed-8a7a-d590d2ef53a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.481150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed69b1e-7dbc-4fed-8a7a-d590d2ef53a0 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.481440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed69b1e-7dbc-4fed-8a7a-d590d2ef53a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.481722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed69b1e-7dbc-4fed-8a7a-d590d2ef53a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.482474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed69b1e-7dbc-4fed-8a7a-d590d2ef53a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.484056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed69b1e-7dbc-4fed-8a7a-d590d2ef53a0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.498552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 798/1597] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.498552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88c181-6adb-45a8-a1c7-854ad48ce707 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.498552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88c181-6adb-45a8-a1c7-854ad48ce707 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.498552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88c181-6adb-45a8-a1c7-854ad48ce707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.504548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:53.515727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-3f88c181-6adb-45a8-a1c7-854ad48ce707 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:53.516935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3f88c181-6adb-45a8-a1c7-854ad48ce707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:53.525540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 799/1598] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:53.527008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88c181-6adb-45a8-a1c7-854ad48ce707 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.535662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88c181-6adb-45a8-a1c7-854ad48ce707 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.542196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1acc92-b3ee-4ae6-9aa7-b3ec734f98fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.542877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1acc92-b3ee-4ae6-9aa7-b3ec734f98fb admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.543171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1acc92-b3ee-4ae6-9aa7-b3ec734f98fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.543477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1acc92-b3ee-4ae6-9aa7-b3ec734f98fb admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.544175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1acc92-b3ee-4ae6-9aa7-b3ec734f98fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.545787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1acc92-b3ee-4ae6-9aa7-b3ec734f98fb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.556287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 800/1599] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.557524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce904a4e-d0f8-4b21-a876-77e3a903dfd3 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.557835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce904a4e-d0f8-4b21-a876-77e3a903dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.558190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce904a4e-d0f8-4b21-a876-77e3a903dfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.575440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ce904a4e-d0f8-4b21-a876-77e3a903dfd3 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:53.576654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ce904a4e-d0f8-4b21-a876-77e3a903dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:53.578861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 800/1600] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:53 2022] PUT /identity/v3/projects/d57676cc6e9c453aad20204f6c6f33de/users/cfc4c09b0a8a4a5db9a4d330250fb2e2/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:53.579824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaad414-889d-4621-87b6-d8d0a3df4a55 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.639149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 801/1601] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:53.640066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce904a4e-d0f8-4b21-a876-77e3a903dfd3 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.645088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaad414-889d-4621-87b6-d8d0a3df4a55 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.649780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce904a4e-d0f8-4b21-a876-77e3a903dfd3 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.650655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282be00-3d69-4388-9f3e-8da9caa20d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.651562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282be00-3d69-4388-9f3e-8da9caa20d47 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.651819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282be00-3d69-4388-9f3e-8da9caa20d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.652100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282be00-3d69-4388-9f3e-8da9caa20d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.652832 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282be00-3d69-4388-9f3e-8da9caa20d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.654483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282be00-3d69-4388-9f3e-8da9caa20d47 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.655410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8770a6-d6a4-43f3-91dd-aa64e7451be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.656083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8770a6-d6a4-43f3-91dd-aa64e7451be5 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.656372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8770a6-d6a4-43f3-91dd-aa64e7451be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.656654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8770a6-d6a4-43f3-91dd-aa64e7451be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.657385 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8770a6-d6a4-43f3-91dd-aa64e7451be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.659009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8770a6-d6a4-43f3-91dd-aa64e7451be5 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.664783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 801/1602] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.665570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282be00-3d69-4388-9f3e-8da9caa20d47 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.672051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 802/1603] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.672051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8770a6-d6a4-43f3-91dd-aa64e7451be5 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.677466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282be00-3d69-4388-9f3e-8da9caa20d47 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.679767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8770a6-d6a4-43f3-91dd-aa64e7451be5 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.682188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ff6ca-9e06-4f1b-8eee-a607b8f5cf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.683066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ff6ca-9e06-4f1b-8eee-a607b8f5cf23 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.683371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ff6ca-9e06-4f1b-8eee-a607b8f5cf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.683629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ff6ca-9e06-4f1b-8eee-a607b8f5cf23 admin admin] PATH_INFO: `/v3/projects/6c247c5f4fb74c78baaf3f4a63a04e42/users/95c6d7130d6c4ae4b8460807e6820ef1/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.684345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a3356-50ba-43b3-8b66-e266a50d11f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.685204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a3356-50ba-43b3-8b66-e266a50d11f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.685493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a3356-50ba-43b3-8b66-e266a50d11f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.685876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a3356-50ba-43b3-8b66-e266a50d11f8 admin admin] PATH_INFO: `/v3/projects/508fd5c51f544a3db6d0b87e37750e01/users/81ca45714ee44a549fd002b4aa13973b/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.699152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ff6ca-9e06-4f1b-8eee-a607b8f5cf23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c247c5f4fb74c78baaf3f4a63a04e42, user_id=95c6d7130d6c4ae4b8460807e6820ef1, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.701844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ff6ca-9e06-4f1b-8eee-a607b8f5cf23 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.702307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a3356-50ba-43b3-8b66-e266a50d11f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=508fd5c51f544a3db6d0b87e37750e01, user_id=81ca45714ee44a549fd002b4aa13973b, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.704950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a3356-50ba-43b3-8b66-e266a50d11f8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.762236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:53.766919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 802/1604] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:53 2022] PUT /identity/v3/projects/6c247c5f4fb74c78baaf3f4a63a04e42/users/95c6d7130d6c4ae4b8460807e6820ef1/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:53.767700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ff6ca-9e06-4f1b-8eee-a607b8f5cf23 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.790401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 803/1605] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:53 2022] PUT /identity/v3/projects/508fd5c51f544a3db6d0b87e37750e01/users/81ca45714ee44a549fd002b4aa13973b/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:53.791239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a3356-50ba-43b3-8b66-e266a50d11f8 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.824799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ff6ca-9e06-4f1b-8eee-a607b8f5cf23 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.853934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a3356-50ba-43b3-8b66-e266a50d11f8 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.858835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5396a-0a4d-47d4-a314-67b39fd61c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.864186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5396a-0a4d-47d4-a314-67b39fd61c3c admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.864186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5396a-0a4d-47d4-a314-67b39fd61c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.864186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5396a-0a4d-47d4-a314-67b39fd61c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.864186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5396a-0a4d-47d4-a314-67b39fd61c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.864186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5396a-0a4d-47d4-a314-67b39fd61c3c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.872659 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd4085-7a45-4421-b92b-2133576069d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.873308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4085-7a45-4421-b92b-2133576069d3 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.873594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4085-7a45-4421-b92b-2133576069d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.873876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 804/1606] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.874015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4085-7a45-4421-b92b-2133576069d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.874681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5396a-0a4d-47d4-a314-67b39fd61c3c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.884816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5396a-0a4d-47d4-a314-67b39fd61c3c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.885020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd4085-7a45-4421-b92b-2133576069d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.886538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd4085-7a45-4421-b92b-2133576069d3 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.889398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fe940-d99b-48b7-abf3-99f5453ce373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.890063 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fe940-d99b-48b7-abf3-99f5453ce373 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.890345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fe940-d99b-48b7-abf3-99f5453ce373 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.890615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fe940-d99b-48b7-abf3-99f5453ce373 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.891321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fe940-d99b-48b7-abf3-99f5453ce373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.892875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fe940-d99b-48b7-abf3-99f5453ce373 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.896358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 803/1607] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:53.897160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd4085-7a45-4421-b92b-2133576069d3 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.905881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 805/1608] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.905881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fe940-d99b-48b7-abf3-99f5453ce373 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.906286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd4085-7a45-4421-b92b-2133576069d3 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.909743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a283a-15aa-48c4-a444-cce0827c7b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.910607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a283a-15aa-48c4-a444-cce0827c7b48 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.910948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a283a-15aa-48c4-a444-cce0827c7b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.911160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a283a-15aa-48c4-a444-cce0827c7b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.915827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fe940-d99b-48b7-abf3-99f5453ce373 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.920440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba2f23-9261-499f-9d40-7df6674f0206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.921106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba2f23-9261-499f-9d40-7df6674f0206 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.921268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a283a-15aa-48c4-a444-cce0827c7b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.921433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba2f23-9261-499f-9d40-7df6674f0206 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.921667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba2f23-9261-499f-9d40-7df6674f0206 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.922424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba2f23-9261-499f-9d40-7df6674f0206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.922557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a283a-15aa-48c4-a444-cce0827c7b48 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.924008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba2f23-9261-499f-9d40-7df6674f0206 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.934283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 804/1609] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:53.935112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a283a-15aa-48c4-a444-cce0827c7b48 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:53.939445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 806/1610] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:53.939445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061cf8e-1706-4cf3-8410-862bd46fdb4e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.939445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061cf8e-1706-4cf3-8410-862bd46fdb4e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.939445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061cf8e-1706-4cf3-8410-862bd46fdb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.946309 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a283a-15aa-48c4-a444-cce0827c7b48 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:53.951765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021bc50-06b6-4d2e-88d6-26c3f816ee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:53.952516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021bc50-06b6-4d2e-88d6-26c3f816ee46 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.952805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021bc50-06b6-4d2e-88d6-26c3f816ee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.953084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021bc50-06b6-4d2e-88d6-26c3f816ee46 admin admin] PATH_INFO: `/v3/projects/1da80740a14a492ca8c47533a8bcb213/users/57b3ef9b8a634073918f572426b3288f/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.955556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0061cf8e-1706-4cf3-8410-862bd46fdb4e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:53.956771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0061cf8e-1706-4cf3-8410-862bd46fdb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:53.965495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 807/1611] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:53.966600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be27ddd-c589-4e1d-b220-0fb91c14e248 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:53.966885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be27ddd-c589-4e1d-b220-0fb91c14e248 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:53.967155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be27ddd-c589-4e1d-b220-0fb91c14e248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:53.973770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021bc50-06b6-4d2e-88d6-26c3f816ee46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da80740a14a492ca8c47533a8bcb213, user_id=57b3ef9b8a634073918f572426b3288f, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:53.976824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021bc50-06b6-4d2e-88d6-26c3f816ee46 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:53.984521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6be27ddd-c589-4e1d-b220-0fb91c14e248 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:53.985705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6be27ddd-c589-4e1d-b220-0fb91c14e248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:53.994161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 808/1612] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:53.995013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be27ddd-c589-4e1d-b220-0fb91c14e248 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:54.006349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be27ddd-c589-4e1d-b220-0fb91c14e248 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:54.011544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc46f5-c156-4dc4-ade6-c5f1f5973c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:54.012487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc46f5-c156-4dc4-ade6-c5f1f5973c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.012707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc46f5-c156-4dc4-ade6-c5f1f5973c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.012979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc46f5-c156-4dc4-ade6-c5f1f5973c5e admin admin] PATH_INFO: `/v3/projects/975f7cf0a1f5445cb45ea441b3c056d5/users/c4fd1155ff11417db8c647f0aadf74a1/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.024112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:54.028111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc46f5-c156-4dc4-ade6-c5f1f5973c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=975f7cf0a1f5445cb45ea441b3c056d5, user_id=c4fd1155ff11417db8c647f0aadf74a1, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:54.030814 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc46f5-c156-4dc4-ade6-c5f1f5973c5e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:54.091494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 805/1613] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:53 2022] PUT /identity/v3/projects/1da80740a14a492ca8c47533a8bcb213/users/57b3ef9b8a634073918f572426b3288f/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:54.092235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021bc50-06b6-4d2e-88d6-26c3f816ee46 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:54.118253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 809/1614] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:53 2022] PUT /identity/v3/projects/975f7cf0a1f5445cb45ea441b3c056d5/users/c4fd1155ff11417db8c647f0aadf74a1/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:54.118954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc46f5-c156-4dc4-ade6-c5f1f5973c5e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:54.171613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021bc50-06b6-4d2e-88d6-26c3f816ee46 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:54.189211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc46f5-c156-4dc4-ade6-c5f1f5973c5e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:54.195259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee679c-a968-4a8e-b451-ac556a3e0e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:54.196157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee679c-a968-4a8e-b451-ac556a3e0e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.196438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee679c-a968-4a8e-b451-ac556a3e0e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.200611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee679c-a968-4a8e-b451-ac556a3e0e91 admin admin] PATH_INFO: `/v3/projects/66a4209b352d49e68dcbc249a7109329/users/f24bf4773a514a8ba237fc6cc609ab2f/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.201584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:54.214003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee679c-a968-4a8e-b451-ac556a3e0e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a4209b352d49e68dcbc249a7109329, user_id=f24bf4773a514a8ba237fc6cc609ab2f, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:54.216905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee679c-a968-4a8e-b451-ac556a3e0e91 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:54.231630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe065c1-8a56-4e79-b432-b39e34866a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:54.232529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe065c1-8a56-4e79-b432-b39e34866a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.232811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe065c1-8a56-4e79-b432-b39e34866a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.233091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe065c1-8a56-4e79-b432-b39e34866a20 admin admin] PATH_INFO: `/v3/projects/d57676cc6e9c453aad20204f6c6f33de/users/cfc4c09b0a8a4a5db9a4d330250fb2e2/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.235200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe065c1-8a56-4e79-b432-b39e34866a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57676cc6e9c453aad20204f6c6f33de, user_id=cfc4c09b0a8a4a5db9a4d330250fb2e2, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:54.237790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe065c1-8a56-4e79-b432-b39e34866a20 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:54.248642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 806/1615] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:54 2022] PUT /identity/v3/projects/d57676cc6e9c453aad20204f6c6f33de/users/cfc4c09b0a8a4a5db9a4d330250fb2e2/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:54.253110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe065c1-8a56-4e79-b432-b39e34866a20 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:54.253110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9fe065c1-8a56-4e79-b432-b39e34866a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:54.310852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe065c1-8a56-4e79-b432-b39e34866a20 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:54.314690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d89fad-11f1-4fc5-8f09-57a2afebed55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:54.315350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d89fad-11f1-4fc5-8f09-57a2afebed55 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.315636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d89fad-11f1-4fc5-8f09-57a2afebed55 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.315910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d89fad-11f1-4fc5-8f09-57a2afebed55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.382541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d89fad-11f1-4fc5-8f09-57a2afebed55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:54.383966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d89fad-11f1-4fc5-8f09-57a2afebed55 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:54.393903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 807/1616] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:54.394731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d89fad-11f1-4fc5-8f09-57a2afebed55 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:54.402576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d89fad-11f1-4fc5-8f09-57a2afebed55 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:54.406720 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d287f-355a-4862-b74e-55e0e24498d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:54.407098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d287f-355a-4862-b74e-55e0e24498d5 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.407378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d287f-355a-4862-b74e-55e0e24498d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.408018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d287f-355a-4862-b74e-55e0e24498d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.425054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d287f-355a-4862-b74e-55e0e24498d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:54.425054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d287f-355a-4862-b74e-55e0e24498d5 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:54.425353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 810/1617] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:54 2022] PUT /identity/v3/projects/66a4209b352d49e68dcbc249a7109329/users/f24bf4773a514a8ba237fc6cc609ab2f/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:54:54.425575 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c8b97-e475-43d2-a13a-b18699c2ccce None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.430419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c8b97-e475-43d2-a13a-b18699c2ccce None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.430834 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c8b97-e475-43d2-a13a-b18699c2ccce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.434243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 808/1618] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:54.435404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2870d-283a-4fbb-8d4d-46b60bda4610 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.435732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2870d-283a-4fbb-8d4d-46b60bda4610 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.435976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2870d-283a-4fbb-8d4d-46b60bda4610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.460132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-bcd2870d-283a-4fbb-8d4d-46b60bda4610 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:54.461736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-bcd2870d-283a-4fbb-8d4d-46b60bda4610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:54.466705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-7a8c8b97-e475-43d2-a13a-b18699c2ccce None None] MFA Rules not processed for user `57b3ef9b8a634073918f572426b3288f`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:54.467922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7a8c8b97-e475-43d2-a13a-b18699c2ccce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:54.533994 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:54.544274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 809/1619] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:54.545702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 811/1620] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:54.545854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae55b8-b753-494b-a23c-f9b5c0d2f2be None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.546242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae55b8-b753-494b-a23c-f9b5c0d2f2be None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.546594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae55b8-b753-494b-a23c-f9b5c0d2f2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.547786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c3a99-e617-4407-bd56-303f0389f392 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.548364 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c3a99-e617-4407-bd56-303f0389f392 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.548847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c3a99-e617-4407-bd56-303f0389f392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.577695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-24ae55b8-b753-494b-a23c-f9b5c0d2f2be None None] MFA Rules not processed for user `cfc4c09b0a8a4a5db9a4d330250fb2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:54.579173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-24ae55b8-b753-494b-a23c-f9b5c0d2f2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:54.580043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-c62c3a99-e617-4407-bd56-303f0389f392 None None] MFA Rules not processed for user `f24bf4773a514a8ba237fc6cc609ab2f`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:54.581237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c62c3a99-e617-4407-bd56-303f0389f392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:54.654967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:54.673545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 810/1621] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:54.674859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae55b8-b753-494b-a23c-f9b5c0d2f2be None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:54.685225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 812/1622] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:54.685225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c3a99-e617-4407-bd56-303f0389f392 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:54.694549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae55b8-b753-494b-a23c-f9b5c0d2f2be None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:54.704538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7416bd-8464-4efc-80f6-7a7cafdc34c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:54.708611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7416bd-8464-4efc-80f6-7a7cafdc34c4 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.709061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7416bd-8464-4efc-80f6-7a7cafdc34c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.709061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7416bd-8464-4efc-80f6-7a7cafdc34c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.709790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7416bd-8464-4efc-80f6-7a7cafdc34c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:54.712320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7416bd-8464-4efc-80f6-7a7cafdc34c4 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:54.792471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c3a99-e617-4407-bd56-303f0389f392 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:54.805547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bed22-870e-46c4-a13d-5ad651c560b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:54.805547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bed22-870e-46c4-a13d-5ad651c560b1 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.805547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bed22-870e-46c4-a13d-5ad651c560b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.805547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bed22-870e-46c4-a13d-5ad651c560b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.815864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bed22-870e-46c4-a13d-5ad651c560b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:54.817711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bed22-870e-46c4-a13d-5ad651c560b1 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:54.836331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 813/1623] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:54.837373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bed22-870e-46c4-a13d-5ad651c560b1 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:54.866277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 811/1624] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:54:54 2022] POST /identity/v3/projects => generated 399 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:54.868508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e129f3-2ee0-4461-9a2c-82fd458f356f None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.869094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e129f3-2ee0-4461-9a2c-82fd458f356f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.869230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e129f3-2ee0-4461-9a2c-82fd458f356f None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.870695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 812/1625] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jul 14 15:54:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:54:54.871263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e129f3-2ee0-4461-9a2c-82fd458f356f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:54.936077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bed22-870e-46c4-a13d-5ad651c560b1 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:54.956811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e129f3-2ee0-4461-9a2c-82fd458f356f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:54.963000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992ef59-703e-460d-aecd-f2b0c2c0dcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:54.963723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992ef59-703e-460d-aecd-f2b0c2c0dcae admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:54.964007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992ef59-703e-460d-aecd-f2b0c2c0dcae admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:54.964280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992ef59-703e-460d-aecd-f2b0c2c0dcae admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:54.964977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992ef59-703e-460d-aecd-f2b0c2c0dcae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:54.967001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992ef59-703e-460d-aecd-f2b0c2c0dcae admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:55.011189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2c9b4-1ad6-466f-abbf-4d17e9d761b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:55.012172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2c9b4-1ad6-466f-abbf-4d17e9d761b6 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.012744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2c9b4-1ad6-466f-abbf-4d17e9d761b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.013199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2c9b4-1ad6-466f-abbf-4d17e9d761b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.071418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 813/1626] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:54:54 2022] POST /identity/v3/users => generated 306 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:54:55.071418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04e622-e57c-4ad1-a3e7-15d844fa83d5 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.071418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04e622-e57c-4ad1-a3e7-15d844fa83d5 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.071418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04e622-e57c-4ad1-a3e7-15d844fa83d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.120548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2c9b4-1ad6-466f-abbf-4d17e9d761b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:55.122876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2c9b4-1ad6-466f-abbf-4d17e9d761b6 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:55.144690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8e04e622-e57c-4ad1-a3e7-15d844fa83d5 None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:55.146287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8e04e622-e57c-4ad1-a3e7-15d844fa83d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:55.150558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:55.166598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 814/1627] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:55.168904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2c9b4-1ad6-466f-abbf-4d17e9d761b6 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:55.193538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:55.196681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2c9b4-1ad6-466f-abbf-4d17e9d761b6 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:55.218552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c781e78-3b64-4399-9e67-c726d6965b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:55.218552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c781e78-3b64-4399-9e67-c726d6965b3b admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.218552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c781e78-3b64-4399-9e67-c726d6965b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.218552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c781e78-3b64-4399-9e67-c726d6965b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.219102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c781e78-3b64-4399-9e67-c726d6965b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:55.228280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c781e78-3b64-4399-9e67-c726d6965b3b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:55.248500 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 815/1628] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:54:55.252379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c781e78-3b64-4399-9e67-c726d6965b3b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:55.270107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 814/1629] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jul 14 15:54:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:55.277840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04e622-e57c-4ad1-a3e7-15d844fa83d5 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:55.283596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c781e78-3b64-4399-9e67-c726d6965b3b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:55.300707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04e622-e57c-4ad1-a3e7-15d844fa83d5 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:55.305243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c991-db80-406b-9ad5-8cb4c53dcf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:55.307801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c991-db80-406b-9ad5-8cb4c53dcf4c admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.310700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c991-db80-406b-9ad5-8cb4c53dcf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.311183 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c991-db80-406b-9ad5-8cb4c53dcf4c admin admin] PATH_INFO: `/v3/projects/d2898df2863f4e93bae635ecf9bacccd/users/483a02a30c1341e790898f47cc79caa3/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.318004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719eacd-eeaf-4b5c-b60f-de4a248cabef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:55.319147 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719eacd-eeaf-4b5c-b60f-de4a248cabef service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.319563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719eacd-eeaf-4b5c-b60f-de4a248cabef service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.320661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719eacd-eeaf-4b5c-b60f-de4a248cabef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.356822 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c991-db80-406b-9ad5-8cb4c53dcf4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2898df2863f4e93bae635ecf9bacccd, user_id=483a02a30c1341e790898f47cc79caa3, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:55.361121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c991-db80-406b-9ad5-8cb4c53dcf4c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:55.413700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719eacd-eeaf-4b5c-b60f-de4a248cabef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:55.415738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719eacd-eeaf-4b5c-b60f-de4a248cabef service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:55.426601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 815/1630] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:54:55.513253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 816/1631] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:55 2022] PUT /identity/v3/projects/d2898df2863f4e93bae635ecf9bacccd/users/483a02a30c1341e790898f47cc79caa3/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:54:55.517517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719eacd-eeaf-4b5c-b60f-de4a248cabef service nova] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:55.530676 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c991-db80-406b-9ad5-8cb4c53dcf4c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:55.533093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719eacd-eeaf-4b5c-b60f-de4a248cabef service nova] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:55.630523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269dbc9c-cc37-4000-b2c4-84ed192ae200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:55.631277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dbc9c-cc37-4000-b2c4-84ed192ae200 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.631568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dbc9c-cc37-4000-b2c4-84ed192ae200 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.632187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269dbc9c-cc37-4000-b2c4-84ed192ae200 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.634251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269dbc9c-cc37-4000-b2c4-84ed192ae200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:55.636441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269dbc9c-cc37-4000-b2c4-84ed192ae200 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:55.651976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 816/1632] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:54:55.655818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c991-db80-406b-9ad5-8cb4c53dcf4c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:55.661501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269dbc9c-cc37-4000-b2c4-84ed192ae200 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:55.662498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb5acc-a626-47e3-a49f-2099c0d68579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:55.663271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb5acc-a626-47e3-a49f-2099c0d68579 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.664161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb5acc-a626-47e3-a49f-2099c0d68579 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.664514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb5acc-a626-47e3-a49f-2099c0d68579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.688873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269dbc9c-cc37-4000-b2c4-84ed192ae200 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:55.706652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdd4c4-600a-4635-a325-47d5f8f1f952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:55.708123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdd4c4-600a-4635-a325-47d5f8f1f952 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.708420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdd4c4-600a-4635-a325-47d5f8f1f952 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.708705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdd4c4-600a-4635-a325-47d5f8f1f952 admin admin] PATH_INFO: `/v3/projects/d2898df2863f4e93bae635ecf9bacccd/users/483a02a30c1341e790898f47cc79caa3/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.717745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdd4c4-600a-4635-a325-47d5f8f1f952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2898df2863f4e93bae635ecf9bacccd, user_id=483a02a30c1341e790898f47cc79caa3, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:55.727202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdd4c4-600a-4635-a325-47d5f8f1f952 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:55.767333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 817/1633] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:54:55 2022] PUT /identity/v3/projects/d2898df2863f4e93bae635ecf9bacccd/users/483a02a30c1341e790898f47cc79caa3/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:54:55.767333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab8240-e966-4930-9f9d-92c99d5f7ba0 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.767333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab8240-e966-4930-9f9d-92c99d5f7ba0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.767333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab8240-e966-4930-9f9d-92c99d5f7ba0 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.767333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 818/1634] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jul 14 15:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:54:55.769870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e373-b331-46d0-9634-cc43a1b3c8e0 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.770258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e373-b331-46d0-9634-cc43a1b3c8e0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.770958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9e373-b331-46d0-9634-cc43a1b3c8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.799419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb5acc-a626-47e3-a49f-2099c0d68579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:55.800585 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb5acc-a626-47e3-a49f-2099c0d68579 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:55.808552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b1f9e373-b331-46d0-9634-cc43a1b3c8e0 None None] MFA Rules not processed for user `483a02a30c1341e790898f47cc79caa3`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:55.809846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b1f9e373-b331-46d0-9634-cc43a1b3c8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:55.829945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 817/1635] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:54:55.835190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01abefa-86a2-465d-8c3f-79be76a7c4c0 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:55.835190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01abefa-86a2-465d-8c3f-79be76a7c4c0 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:55.836674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01abefa-86a2-465d-8c3f-79be76a7c4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:55.865255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e01abefa-86a2-465d-8c3f-79be76a7c4c0 None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:55.869933 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e01abefa-86a2-465d-8c3f-79be76a7c4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:55.922006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:55.930486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 819/1636] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:54:55 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:55.939297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e373-b331-46d0-9634-cc43a1b3c8e0 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:56.020191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 818/1637] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:54:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:56.030990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9e373-b331-46d0-9634-cc43a1b3c8e0 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:56.043515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d0e1-abad-44e4-9c84-8ab6845f97a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:56.046418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d0e1-abad-44e4-9c84-8ab6845f97a2 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:56.046896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d0e1-abad-44e4-9c84-8ab6845f97a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:56.047322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d0e1-abad-44e4-9c84-8ab6845f97a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:56.058999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0d0e1-abad-44e4-9c84-8ab6845f97a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:56.060495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0d0e1-abad-44e4-9c84-8ab6845f97a2 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:56.070901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 820/1638] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:55 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:54:56.205032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01abefa-86a2-465d-8c3f-79be76a7c4c0 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:56.231185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01abefa-86a2-465d-8c3f-79be76a7c4c0 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:56.236958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50167042-2bbc-47c7-ac1a-49a08fb25087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:56.238459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167042-2bbc-47c7-ac1a-49a08fb25087 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:56.238804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167042-2bbc-47c7-ac1a-49a08fb25087 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:56.239502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50167042-2bbc-47c7-ac1a-49a08fb25087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:56.258811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50167042-2bbc-47c7-ac1a-49a08fb25087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:56.260660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50167042-2bbc-47c7-ac1a-49a08fb25087 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:56.266854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:54:56.275966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 819/1639] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:56.698235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d0e1-abad-44e4-9c84-8ab6845f97a2 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:56.709990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d0e1-abad-44e4-9c84-8ab6845f97a2 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:56.717357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3478525-5ef4-4cdc-8bae-58e94fbce322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:56.722111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3478525-5ef4-4cdc-8bae-58e94fbce322 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:56.722111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3478525-5ef4-4cdc-8bae-58e94fbce322 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:56.722111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3478525-5ef4-4cdc-8bae-58e94fbce322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:56.838932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3478525-5ef4-4cdc-8bae-58e94fbce322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:56.840407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3478525-5ef4-4cdc-8bae-58e94fbce322 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:56.855870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 821/1640] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:54:58.531392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a675f-1270-4392-8fc5-c871926a415d None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:58.531392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a675f-1270-4392-8fc5-c871926a415d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:58.531392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a675f-1270-4392-8fc5-c871926a415d None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:58.531392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 820/1641] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jul 14 15:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:54:58.543594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e2e68-1a76-4ecb-9099-48f80f1a34cc None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:58.544566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e2e68-1a76-4ecb-9099-48f80f1a34cc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:58.545042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e2e68-1a76-4ecb-9099-48f80f1a34cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:58.573194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-de7e2e68-1a76-4ecb-9099-48f80f1a34cc None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:58.575152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-de7e2e68-1a76-4ecb-9099-48f80f1a34cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:58.591266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b742d-7ede-46b4-8fbe-ca1e24b4475c None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:58.605746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b742d-7ede-46b4-8fbe-ca1e24b4475c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:58.605746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b742d-7ede-46b4-8fbe-ca1e24b4475c None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:58.605746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 821/1642] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jul 14 15:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:54:58.605746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d09d4e-9699-427f-a1af-6b3516e780fb None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:58.605746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d09d4e-9699-427f-a1af-6b3516e780fb None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:58.605746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d09d4e-9699-427f-a1af-6b3516e780fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:58.606302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 822/1643] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jul 14 15:54:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:54:58.619534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a9d09d4e-9699-427f-a1af-6b3516e780fb None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:54:58.620852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a9d09d4e-9699-427f-a1af-6b3516e780fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:54:58.637200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 822/1644] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:54:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:54:58.664350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e2e68-1a76-4ecb-9099-48f80f1a34cc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:54:58.677262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e2e68-1a76-4ecb-9099-48f80f1a34cc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:54:58.683199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a3789-0a64-4656-9ea4-074dfec3de6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:54:58.684093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a3789-0a64-4656-9ea4-074dfec3de6c service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:54:58.684662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a3789-0a64-4656-9ea4-074dfec3de6c service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:54:58.685640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a3789-0a64-4656-9ea4-074dfec3de6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:54:58.710955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a3789-0a64-4656-9ea4-074dfec3de6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:54:58.713191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a3789-0a64-4656-9ea4-074dfec3de6c service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:54:58.737567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 823/1645] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:54:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:02.815672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d09d4e-9699-427f-a1af-6b3516e780fb None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:03.010890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d09d4e-9699-427f-a1af-6b3516e780fb None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:03.017791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:03.019525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:03.020006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:03.020638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] PATH_INFO: `/v3/users/f24bf4773a514a8ba237fc6cc609ab2f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:03.022478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f24bf4773a514a8ba237fc6cc609ab2f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:03.024798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:03.289232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f24bf4773a514a8ba237fc6cc609ab2f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:03.411641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f24bf4773a514a8ba237fc6cc609ab2f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:03.412553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f24bf4773a514a8ba237fc6cc609ab2f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:03.426017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f24bf4773a514a8ba237fc6cc609ab2f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:03.427050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f24bf4773a514a8ba237fc6cc609ab2f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:03.469670 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 823/1646] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:02 2022] DELETE /identity/v3/users/f24bf4773a514a8ba237fc6cc609ab2f => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:03.474235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37787a5e-a0c1-4f19-9f11-1d508940f005 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:03.474751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37787a5e-a0c1-4f19-9f11-1d508940f005 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:03.474928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37787a5e-a0c1-4f19-9f11-1d508940f005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:03.499978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-37787a5e-a0c1-4f19-9f11-1d508940f005 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:03.503318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-37787a5e-a0c1-4f19-9f11-1d508940f005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:03.629045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:03.641390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 824/1647] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:03.803917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:03.806635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:03.972788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2a409-4e4c-4a41-b39e-0fa1d5d34a86 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:03.980658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa5ade-70b6-4add-95b7-32967f3644d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:03.981354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa5ade-70b6-4add-95b7-32967f3644d1 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:03.981647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa5ade-70b6-4add-95b7-32967f3644d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:03.985148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa5ade-70b6-4add-95b7-32967f3644d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:04.016480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa5ade-70b6-4add-95b7-32967f3644d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:04.019657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa5ade-70b6-4add-95b7-32967f3644d1 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:04.035058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 824/1648] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:05.252831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dae5eb-49d3-41f5-8e75-4ff88da78989 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:05.253334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dae5eb-49d3-41f5-8e75-4ff88da78989 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:05.253571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dae5eb-49d3-41f5-8e75-4ff88da78989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:05.313060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b4dae5eb-49d3-41f5-8e75-4ff88da78989 None None] MFA Rules not processed for user `dcdbbbf1b9944b09898fa8731184c845`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:05.314631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b4dae5eb-49d3-41f5-8e75-4ff88da78989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:05.461032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 825/1649] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:05.477615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa5ade-70b6-4add-95b7-32967f3644d1 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:05.479262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d5fa5ade-70b6-4add-95b7-32967f3644d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:05.504049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:05.659656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa5ade-70b6-4add-95b7-32967f3644d1 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:05.699249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b360686-f02e-4116-8ba3-cc654c5818b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:05.701106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b360686-f02e-4116-8ba3-cc654c5818b2 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:05.702710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b360686-f02e-4116-8ba3-cc654c5818b2 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:05.703231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b360686-f02e-4116-8ba3-cc654c5818b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:05.761066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b360686-f02e-4116-8ba3-cc654c5818b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:05.767269 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b360686-f02e-4116-8ba3-cc654c5818b2 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:05.809075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 825/1650] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:06.139400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dae5eb-49d3-41f5-8e75-4ff88da78989 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:06.140952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b4dae5eb-49d3-41f5-8e75-4ff88da78989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:06.165458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dae5eb-49d3-41f5-8e75-4ff88da78989 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:06.173697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:06.174779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:06.175312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:06.175699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] PATH_INFO: `/v3/projects/66a4209b352d49e68dcbc249a7109329` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:06.177807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a4209b352d49e68dcbc249a7109329)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:06.181028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:06.382519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a4209b352d49e68dcbc249a7109329', 'request_id': 'req-d8f23960-7ae4-40bd-a404-071d03216fc8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:06.392427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a4209b352d49e68dcbc249a7109329', 'request_id': 'req-d8f23960-7ae4-40bd-a404-071d03216fc8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:06.481940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a4209b352d49e68dcbc249a7109329', 'request_id': 'req-d8f23960-7ae4-40bd-a404-071d03216fc8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:06.513605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] The token cache is being invalidate because project 66a4209b352d49e68dcbc249a7109329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:06.515300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:06.517230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d8f23960-7ae4-40bd-a404-071d03216fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:06.524223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 826/1651] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:06 2022] DELETE /identity/v3/projects/66a4209b352d49e68dcbc249a7109329 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:06.526022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923377-3b21-4f12-9f52-dee097d1c860 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:06.526461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923377-3b21-4f12-9f52-dee097d1c860 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:06.526600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923377-3b21-4f12-9f52-dee097d1c860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:06.568151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-8b923377-3b21-4f12-9f52-dee097d1c860 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:06.568151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8b923377-3b21-4f12-9f52-dee097d1c860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:06.752662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 826/1652] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:06.759871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495ee1f-de92-4f35-aa7c-5e71d15cd67f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:06.760547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495ee1f-de92-4f35-aa7c-5e71d15cd67f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:06.761296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495ee1f-de92-4f35-aa7c-5e71d15cd67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:06.825918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0495ee1f-de92-4f35-aa7c-5e71d15cd67f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:06.827798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0495ee1f-de92-4f35-aa7c-5e71d15cd67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:06.853095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 827/1653] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:06.856994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b923377-3b21-4f12-9f52-dee097d1c860 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:06.877881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b923377-3b21-4f12-9f52-dee097d1c860 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:06.888575 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12720b-3de1-4b70-9f54-5cad6d65a00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:06.890047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12720b-3de1-4b70-9f54-5cad6d65a00c admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:06.892404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12720b-3de1-4b70-9f54-5cad6d65a00c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:06.892700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12720b-3de1-4b70-9f54-5cad6d65a00c admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:06.893551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12720b-3de1-4b70-9f54-5cad6d65a00c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:55:06.895407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12720b-3de1-4b70-9f54-5cad6d65a00c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:06.901638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12720b-3de1-4b70-9f54-5cad6d65a00c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:06.940060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 827/1654] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:55:06.945769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495ee1f-de92-4f35-aa7c-5e71d15cd67f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:06.958775 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495ee1f-de92-4f35-aa7c-5e71d15cd67f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:06.979020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd186d48-9249-4d66-8099-0be22999bdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:06.980826 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd186d48-9249-4d66-8099-0be22999bdbd admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:06.981120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd186d48-9249-4d66-8099-0be22999bdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:06.981405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd186d48-9249-4d66-8099-0be22999bdbd admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:06.982354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd186d48-9249-4d66-8099-0be22999bdbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:06.988011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd186d48-9249-4d66-8099-0be22999bdbd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:07.123194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 828/1655] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:06 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:07.123496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12720b-3de1-4b70-9f54-5cad6d65a00c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:07.139255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12720b-3de1-4b70-9f54-5cad6d65a00c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:07.384475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd186d48-9249-4d66-8099-0be22999bdbd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:07.386331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-bd186d48-9249-4d66-8099-0be22999bdbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:07.423181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783747c-e0f6-4e08-b727-151c712a866b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:07.424045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783747c-e0f6-4e08-b727-151c712a866b admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:07.424333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783747c-e0f6-4e08-b727-151c712a866b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:07.424617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783747c-e0f6-4e08-b727-151c712a866b admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:07.425333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783747c-e0f6-4e08-b727-151c712a866b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:07.442340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783747c-e0f6-4e08-b727-151c712a866b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:07.531154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:07.591285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd186d48-9249-4d66-8099-0be22999bdbd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:07.620602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:07.621462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:07.621749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:07.623064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] PATH_INFO: `/v3/users/57b3ef9b8a634073918f572426b3288f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:07.624874 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 828/1656] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:07 2022] POST /identity/v3/users => generated 286 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:07.626113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783747c-e0f6-4e08-b727-151c712a866b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:07.626621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b3ef9b8a634073918f572426b3288f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:07.627735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: WARNING py.warnings [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:55:07.627735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: warnings.warn(msg) Jul 14 15:55:07.627735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: Jul 14 15:55:07.628963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:07.650734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783747c-e0f6-4e08-b727-151c712a866b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:07.657330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950707-f4d3-43e2-b180-e84b1f41bd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:07.658121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950707-f4d3-43e2-b180-e84b1f41bd67 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:07.658410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950707-f4d3-43e2-b180-e84b1f41bd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:07.658691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950707-f4d3-43e2-b180-e84b1f41bd67 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:07.659584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950707-f4d3-43e2-b180-e84b1f41bd67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:07.663130 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950707-f4d3-43e2-b180-e84b1f41bd67 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:07.686577 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 829/1657] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:07.687563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950707-f4d3-43e2-b180-e84b1f41bd67 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:07.710011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:07.729761 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950707-f4d3-43e2-b180-e84b1f41bd67 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:07.755711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18336c1e-3ab7-4226-a53f-a0e91d870675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:07.756590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18336c1e-3ab7-4226-a53f-a0e91d870675 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:07.759530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18336c1e-3ab7-4226-a53f-a0e91d870675 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:07.761696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18336c1e-3ab7-4226-a53f-a0e91d870675 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:07.765771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18336c1e-3ab7-4226-a53f-a0e91d870675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:07.779779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18336c1e-3ab7-4226-a53f-a0e91d870675 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:07.826958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 830/1658] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:07.828811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18336c1e-3ab7-4226-a53f-a0e91d870675 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:07.867682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18336c1e-3ab7-4226-a53f-a0e91d870675 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:07.887104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f256a3-6f8b-480b-96b4-289ad4180090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:07.888307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f256a3-6f8b-480b-96b4-289ad4180090 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:07.888651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f256a3-6f8b-480b-96b4-289ad4180090 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:07.888996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f256a3-6f8b-480b-96b4-289ad4180090 admin admin] PATH_INFO: `/v3/projects/c1b9ef2ffa8b458795820f28f4367791/users/33db7633c6f045b1b22fac32e1cf8125/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:07.928985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f256a3-6f8b-480b-96b4-289ad4180090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b9ef2ffa8b458795820f28f4367791, user_id=33db7633c6f045b1b22fac32e1cf8125, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:07.933216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f256a3-6f8b-480b-96b4-289ad4180090 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:08.048136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 831/1659] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:07 2022] PUT /identity/v3/projects/c1b9ef2ffa8b458795820f28f4367791/users/33db7633c6f045b1b22fac32e1cf8125/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:08.062563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fe5cd-06b7-4aa9-8015-438649c44524 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:08.066908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fe5cd-06b7-4aa9-8015-438649c44524 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:08.066908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fe5cd-06b7-4aa9-8015-438649c44524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:08.099502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b3ef9b8a634073918f572426b3288f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:08.158376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6e7fe5cd-06b7-4aa9-8015-438649c44524 None None] MFA Rules not processed for user `33db7633c6f045b1b22fac32e1cf8125`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:08.159631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6e7fe5cd-06b7-4aa9-8015-438649c44524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:08.247120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b3ef9b8a634073918f572426b3288f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:08.248725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b3ef9b8a634073918f572426b3288f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:08.274945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b3ef9b8a634073918f572426b3288f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:08.275807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-919aa5af-f4cc-4eda-8a99-cd6036c1fb2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b3ef9b8a634073918f572426b3288f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:08.335895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 832/1660] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:08.340154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fe5cd-06b7-4aa9-8015-438649c44524 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:08.340700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 829/1661] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:07 2022] DELETE /identity/v3/users/57b3ef9b8a634073918f572426b3288f => generated 0 bytes in 957 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:08.342154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6e7fe5cd-06b7-4aa9-8015-438649c44524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:08.350309 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e219d99-2f98-4e6b-8b97-e9dda7eb4c36 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:08.350669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e219d99-2f98-4e6b-8b97-e9dda7eb4c36 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:08.351059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e219d99-2f98-4e6b-8b97-e9dda7eb4c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:08.419892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8e219d99-2f98-4e6b-8b97-e9dda7eb4c36 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:08.421494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8e219d99-2f98-4e6b-8b97-e9dda7eb4c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:08.535156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fe5cd-06b7-4aa9-8015-438649c44524 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:08.573706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 830/1662] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:08.662278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8194ac4-2fa5-42fc-91d5-300e9b86a092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:08.664323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8194ac4-2fa5-42fc-91d5-300e9b86a092 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:08.664803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8194ac4-2fa5-42fc-91d5-300e9b86a092 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:08.665219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8194ac4-2fa5-42fc-91d5-300e9b86a092 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:08.666744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8194ac4-2fa5-42fc-91d5-300e9b86a092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:08.672159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e219d99-2f98-4e6b-8b97-e9dda7eb4c36 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:08.672862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8194ac4-2fa5-42fc-91d5-300e9b86a092 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:08.677188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8e219d99-2f98-4e6b-8b97-e9dda7eb4c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:08.825575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e219d99-2f98-4e6b-8b97-e9dda7eb4c36 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:08.838466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb199f-ac3c-4b55-89fe-255e4f6b7b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:08.839287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb199f-ac3c-4b55-89fe-255e4f6b7b15 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:08.839570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb199f-ac3c-4b55-89fe-255e4f6b7b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:08.839990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb199f-ac3c-4b55-89fe-255e4f6b7b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:08.861579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb199f-ac3c-4b55-89fe-255e4f6b7b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:08.863103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb199f-ac3c-4b55-89fe-255e4f6b7b15 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:08.877909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 833/1663] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:08 2022] POST /identity/v3/projects => generated 373 bytes in 535 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:08.879332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8194ac4-2fa5-42fc-91d5-300e9b86a092 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:08.888339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 831/1664] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:08.981494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:09.120224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8194ac4-2fa5-42fc-91d5-300e9b86a092 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:09.133418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf99b2b-5680-4ce7-9f6c-3fa75aac065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:09.135280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf99b2b-5680-4ce7-9f6c-3fa75aac065b admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:09.136775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf99b2b-5680-4ce7-9f6c-3fa75aac065b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:09.138308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf99b2b-5680-4ce7-9f6c-3fa75aac065b admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:09.139191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf99b2b-5680-4ce7-9f6c-3fa75aac065b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:09.145715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf99b2b-5680-4ce7-9f6c-3fa75aac065b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:09.269886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 834/1665] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:08 2022] POST /identity/v3/users => generated 293 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:55:09.271111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb199f-ac3c-4b55-89fe-255e4f6b7b15 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:09.292825 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb199f-ac3c-4b55-89fe-255e4f6b7b15 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:09.312676 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73a508-2696-4f6a-bfdb-71319b988ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:09.313615 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a508-2696-4f6a-bfdb-71319b988ffe admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:09.314083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a508-2696-4f6a-bfdb-71319b988ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:09.314512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a508-2696-4f6a-bfdb-71319b988ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:09.315258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73a508-2696-4f6a-bfdb-71319b988ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:09.317119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73a508-2696-4f6a-bfdb-71319b988ffe admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:09.347574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf99b2b-5680-4ce7-9f6c-3fa75aac065b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:09.353256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 832/1666] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:09.361461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf99b2b-5680-4ce7-9f6c-3fa75aac065b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:09.371840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c74e0-5e65-4c5f-813f-7a78a8b180ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:09.373653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c74e0-5e65-4c5f-813f-7a78a8b180ad admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:09.373653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c74e0-5e65-4c5f-813f-7a78a8b180ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:09.373653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c74e0-5e65-4c5f-813f-7a78a8b180ad admin admin] PATH_INFO: `/v3/projects/f5d29d0ac9fc4ebf82c0d6907cbd689c/users/2a60cc08f2da45ee8241664fc8d868b9/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:09.400280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:09.423380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c74e0-5e65-4c5f-813f-7a78a8b180ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d29d0ac9fc4ebf82c0d6907cbd689c, user_id=2a60cc08f2da45ee8241664fc8d868b9, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:09.426178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c74e0-5e65-4c5f-813f-7a78a8b180ad admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:09.542284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 835/1667] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:09 2022] PUT /identity/v3/projects/f5d29d0ac9fc4ebf82c0d6907cbd689c/users/2a60cc08f2da45ee8241664fc8d868b9/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:09.545065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73a508-2696-4f6a-bfdb-71319b988ffe admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:09.560287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73a508-2696-4f6a-bfdb-71319b988ffe admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:09.739666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd3685-9e6f-4c1a-bd2b-47f2401c99a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:09.745843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd3685-9e6f-4c1a-bd2b-47f2401c99a9 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:09.745843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd3685-9e6f-4c1a-bd2b-47f2401c99a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:09.745843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd3685-9e6f-4c1a-bd2b-47f2401c99a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:09.745843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd3685-9e6f-4c1a-bd2b-47f2401c99a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:09.745843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd3685-9e6f-4c1a-bd2b-47f2401c99a9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:09.789202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 833/1668] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:09.794388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c74e0-5e65-4c5f-813f-7a78a8b180ad admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:09.811258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c74e0-5e65-4c5f-813f-7a78a8b180ad admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:09.824001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6d1de-8121-4c4e-a46b-0f14edd1090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:09.825076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6d1de-8121-4c4e-a46b-0f14edd1090d admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:09.825371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6d1de-8121-4c4e-a46b-0f14edd1090d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:09.825654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6d1de-8121-4c4e-a46b-0f14edd1090d admin admin] PATH_INFO: `/v3/projects/f5d29d0ac9fc4ebf82c0d6907cbd689c/users/2a60cc08f2da45ee8241664fc8d868b9/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:09.830464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6d1de-8121-4c4e-a46b-0f14edd1090d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d29d0ac9fc4ebf82c0d6907cbd689c, user_id=2a60cc08f2da45ee8241664fc8d868b9, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:09.838431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6d1de-8121-4c4e-a46b-0f14edd1090d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:09.891924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 836/1669] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:09 2022] PUT /identity/v3/projects/f5d29d0ac9fc4ebf82c0d6907cbd689c/users/2a60cc08f2da45ee8241664fc8d868b9/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:09.893259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d544fa-a8f3-42c6-a7a7-02774bc2d834 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:09.893862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d544fa-a8f3-42c6-a7a7-02774bc2d834 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:09.894341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d544fa-a8f3-42c6-a7a7-02774bc2d834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:09.977096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c5d544fa-a8f3-42c6-a7a7-02774bc2d834 None None] MFA Rules not processed for user `2a60cc08f2da45ee8241664fc8d868b9`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:09.982182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c5d544fa-a8f3-42c6-a7a7-02774bc2d834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:10.204803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 834/1670] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:10.215362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6d1de-8121-4c4e-a46b-0f14edd1090d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:10.219869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2dc6d1de-8121-4c4e-a46b-0f14edd1090d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:10.425089 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6d1de-8121-4c4e-a46b-0f14edd1090d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:10.439929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08bc14-1e26-4e9f-ae0b-fce681b16c17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:10.440537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08bc14-1e26-4e9f-ae0b-fce681b16c17 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:10.441381 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08bc14-1e26-4e9f-ae0b-fce681b16c17 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:10.441754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08bc14-1e26-4e9f-ae0b-fce681b16c17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:10.463092 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08bc14-1e26-4e9f-ae0b-fce681b16c17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:10.464716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08bc14-1e26-4e9f-ae0b-fce681b16c17 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:10.491925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 837/1671] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:11.166251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d544fa-a8f3-42c6-a7a7-02774bc2d834 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:11.168251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c5d544fa-a8f3-42c6-a7a7-02774bc2d834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:11.365189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d544fa-a8f3-42c6-a7a7-02774bc2d834 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:11.371435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88366648-f430-4b85-a231-c053d431494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:11.374230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88366648-f430-4b85-a231-c053d431494f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:11.375202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88366648-f430-4b85-a231-c053d431494f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:11.375696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88366648-f430-4b85-a231-c053d431494f admin admin] PATH_INFO: `/v3/projects/1da80740a14a492ca8c47533a8bcb213` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:11.377163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88366648-f430-4b85-a231-c053d431494f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da80740a14a492ca8c47533a8bcb213)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:11.389924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88366648-f430-4b85-a231-c053d431494f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:11.619051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-88366648-f430-4b85-a231-c053d431494f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da80740a14a492ca8c47533a8bcb213', 'request_id': 'req-88366648-f430-4b85-a231-c053d431494f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:11.641135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-88366648-f430-4b85-a231-c053d431494f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da80740a14a492ca8c47533a8bcb213', 'request_id': 'req-88366648-f430-4b85-a231-c053d431494f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:11.729097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-88366648-f430-4b85-a231-c053d431494f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da80740a14a492ca8c47533a8bcb213', 'request_id': 'req-88366648-f430-4b85-a231-c053d431494f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:11.758992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-88366648-f430-4b85-a231-c053d431494f admin admin] The token cache is being invalidate because project 1da80740a14a492ca8c47533a8bcb213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:11.759378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-88366648-f430-4b85-a231-c053d431494f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:11.760223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-88366648-f430-4b85-a231-c053d431494f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:11.763631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 835/1672] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:11 2022] DELETE /identity/v3/projects/1da80740a14a492ca8c47533a8bcb213 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:11.774777 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08bc14-1e26-4e9f-ae0b-fce681b16c17 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:11.799132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08bc14-1e26-4e9f-ae0b-fce681b16c17 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:11.924177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:11.938353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:11.950422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:11.950422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:11.950422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:11.950422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] PATH_INFO: `/v3/users/483a02a30c1341e790898f47cc79caa3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:11.951576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=483a02a30c1341e790898f47cc79caa3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:11.953493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:12.046525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88366648-f430-4b85-a231-c053d431494f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:12.047999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-88366648-f430-4b85-a231-c053d431494f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:12.076012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88366648-f430-4b85-a231-c053d431494f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:12.084251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:12.085261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:12.085551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:12.085846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] PATH_INFO: `/v3/users/33db7633c6f045b1b22fac32e1cf8125` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:12.087889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33db7633c6f045b1b22fac32e1cf8125)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:12.091542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:12.311794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '483a02a30c1341e790898f47cc79caa3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.430375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '483a02a30c1341e790898f47cc79caa3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.431060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '483a02a30c1341e790898f47cc79caa3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.446247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '483a02a30c1341e790898f47cc79caa3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.449944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-58698b1a-ffc1-44b6-aa21-6390a7feb3ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '483a02a30c1341e790898f47cc79caa3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.467959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33db7633c6f045b1b22fac32e1cf8125'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.495288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 838/1673] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:11 2022] DELETE /identity/v3/users/483a02a30c1341e790898f47cc79caa3 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:12.550056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33db7633c6f045b1b22fac32e1cf8125'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.553094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33db7633c6f045b1b22fac32e1cf8125'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.567010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33db7633c6f045b1b22fac32e1cf8125'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.572446 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cee981e4-62a3-4550-9b8a-b6d67da8f810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33db7633c6f045b1b22fac32e1cf8125'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:12.596944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bca97-98a7-42e7-a7c1-e4b1516d89e4 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:12.597668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bca97-98a7-42e7-a7c1-e4b1516d89e4 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:12.598059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bca97-98a7-42e7-a7c1-e4b1516d89e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:12.636758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 836/1674] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:12 2022] DELETE /identity/v3/users/33db7633c6f045b1b22fac32e1cf8125 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:55:12.643723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891688e-9480-40fb-a109-ab51f69d2cc7 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:12.644064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891688e-9480-40fb-a109-ab51f69d2cc7 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:12.644353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891688e-9480-40fb-a109-ab51f69d2cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:12.658859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-da2bca97-98a7-42e7-a7c1-e4b1516d89e4 None None] MFA Rules not processed for user `d476a2bec8164b8bbf9a5aaa419a0351`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:12.660117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-da2bca97-98a7-42e7-a7c1-e4b1516d89e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:12.703475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0891688e-9480-40fb-a109-ab51f69d2cc7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:12.706359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0891688e-9480-40fb-a109-ab51f69d2cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:12.847931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 839/1675] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:12.852192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f618280-eec2-4a02-aae8-54aae8c77c6a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:12.852645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f618280-eec2-4a02-aae8-54aae8c77c6a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:12.853043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f618280-eec2-4a02-aae8-54aae8c77c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:12.903712 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 837/1676] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:12.905562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f98543-5e54-4094-9c24-d40f9bf7f4d8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:12.905939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f98543-5e54-4094-9c24-d40f9bf7f4d8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:12.906856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f98543-5e54-4094-9c24-d40f9bf7f4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:12.922295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0f618280-eec2-4a02-aae8-54aae8c77c6a None None] MFA Rules not processed for user `d476a2bec8164b8bbf9a5aaa419a0351`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:12.940666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0f618280-eec2-4a02-aae8-54aae8c77c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:12.961956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 840/1677] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:12.968069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f618280-eec2-4a02-aae8-54aae8c77c6a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:12.970003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0f618280-eec2-4a02-aae8-54aae8c77c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:12.995205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-80f98543-5e54-4094-9c24-d40f9bf7f4d8 None None] MFA Rules not processed for user `c4fd1155ff11417db8c647f0aadf74a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:12.997738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-80f98543-5e54-4094-9c24-d40f9bf7f4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:13.011579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:13.076449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f618280-eec2-4a02-aae8-54aae8c77c6a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:13.081602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:13.095237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c40ea1-583f-4d3b-8c65-0a53350a7bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:13.095935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c40ea1-583f-4d3b-8c65-0a53350a7bad service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:13.096223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c40ea1-583f-4d3b-8c65-0a53350a7bad service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:13.096652 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c40ea1-583f-4d3b-8c65-0a53350a7bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:13.140980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c40ea1-583f-4d3b-8c65-0a53350a7bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:13.147906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c40ea1-583f-4d3b-8c65-0a53350a7bad service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:13.174041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 841/1678] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:13.175154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c40ea1-583f-4d3b-8c65-0a53350a7bad service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:13.177580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-89c40ea1-583f-4d3b-8c65-0a53350a7bad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:13.208692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 838/1679] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:13.212227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cf643-0829-4a2e-9a46-3c1d7b5dff0b None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:13.213973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cf643-0829-4a2e-9a46-3c1d7b5dff0b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:13.216188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cf643-0829-4a2e-9a46-3c1d7b5dff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:13.273760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f30cf643-0829-4a2e-9a46-3c1d7b5dff0b None None] MFA Rules not processed for user `dcdbbbf1b9944b09898fa8731184c845`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:13.283304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f30cf643-0829-4a2e-9a46-3c1d7b5dff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:13.347695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c40ea1-583f-4d3b-8c65-0a53350a7bad service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:13.365735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753daf2-0848-4739-8c0a-3f74b0cd0563 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:13.367795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753daf2-0848-4739-8c0a-3f74b0cd0563 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:13.368662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753daf2-0848-4739-8c0a-3f74b0cd0563 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:13.369470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753daf2-0848-4739-8c0a-3f74b0cd0563 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:13.410014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753daf2-0848-4739-8c0a-3f74b0cd0563 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:13.415299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753daf2-0848-4739-8c0a-3f74b0cd0563 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:13.445776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 842/1680] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:13.447083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd5fc0-3cd0-4be6-8239-e59fd39ae8be None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:13.447387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd5fc0-3cd0-4be6-8239-e59fd39ae8be None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:13.447671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd5fc0-3cd0-4be6-8239-e59fd39ae8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:13.458259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 839/1681] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:13.463623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cf643-0829-4a2e-9a46-3c1d7b5dff0b None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:13.464409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f30cf643-0829-4a2e-9a46-3c1d7b5dff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:13.510651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-04dd5fc0-3cd0-4be6-8239-e59fd39ae8be None None] MFA Rules not processed for user `81ca45714ee44a549fd002b4aa13973b`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:13.512602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-04dd5fc0-3cd0-4be6-8239-e59fd39ae8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:13.516507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cf643-0829-4a2e-9a46-3c1d7b5dff0b None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:13.536751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0b75a-1185-4f3b-af97-7c871f4779f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:13.536751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0b75a-1185-4f3b-af97-7c871f4779f4 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:13.537012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0b75a-1185-4f3b-af97-7c871f4779f4 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:13.537206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0b75a-1185-4f3b-af97-7c871f4779f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:13.561744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0b75a-1185-4f3b-af97-7c871f4779f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:13.563388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0b75a-1185-4f3b-af97-7c871f4779f4 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:13.593917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 840/1682] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:13.783251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 843/1683] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:13.812262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a51e4-485d-43e6-848b-2cc0c3a24b76 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:13.812262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a51e4-485d-43e6-848b-2cc0c3a24b76 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:13.812262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a51e4-485d-43e6-848b-2cc0c3a24b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:13.880816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-fd5a51e4-485d-43e6-848b-2cc0c3a24b76 None None] MFA Rules not processed for user `81ca45714ee44a549fd002b4aa13973b`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:13.882521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fd5a51e4-485d-43e6-848b-2cc0c3a24b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:13.908987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 841/1684] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:14.123408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd5fc0-3cd0-4be6-8239-e59fd39ae8be None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:14.127223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-04dd5fc0-3cd0-4be6-8239-e59fd39ae8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:14.192537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd5fc0-3cd0-4be6-8239-e59fd39ae8be None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:14.217404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:14.219400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:14.219854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:14.220834 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] PATH_INFO: `/v3/projects/d2898df2863f4e93bae635ecf9bacccd` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:14.223341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2898df2863f4e93bae635ecf9bacccd)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:14.227886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: WARNING py.warnings [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 15:55:14.227886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: warnings.warn(msg) Jul 14 15:55:14.227886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: Jul 14 15:55:14.229848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:14.504664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2898df2863f4e93bae635ecf9bacccd', 'request_id': 'req-75733ee9-7711-43c4-be01-5f0445b75a73'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:14.530014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2898df2863f4e93bae635ecf9bacccd', 'request_id': 'req-75733ee9-7711-43c4-be01-5f0445b75a73'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:14.623323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcce0eb-26aa-4953-92a0-1dbc0f570939 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:14.623831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcce0eb-26aa-4953-92a0-1dbc0f570939 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:14.624111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcce0eb-26aa-4953-92a0-1dbc0f570939 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:14.625620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 842/1685] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jul 14 15:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:55:14.630176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2898df2863f4e93bae635ecf9bacccd', 'request_id': 'req-75733ee9-7711-43c4-be01-5f0445b75a73'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:14.646120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c8532-0611-426c-9b37-ed68a714e332 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:14.647068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c8532-0611-426c-9b37-ed68a714e332 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:14.647469 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308c8532-0611-426c-9b37-ed68a714e332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:14.672998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] The token cache is being invalidate because project d2898df2863f4e93bae635ecf9bacccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:14.673936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:14.674955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-75733ee9-7711-43c4-be01-5f0445b75a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:14.684394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 844/1686] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:14 2022] DELETE /identity/v3/projects/d2898df2863f4e93bae635ecf9bacccd => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:14.686397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c449c-88f9-4131-bfff-7a85f62930c8 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:14.686836 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c449c-88f9-4131-bfff-7a85f62930c8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:14.687258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c449c-88f9-4131-bfff-7a85f62930c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:14.719837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-fd9c449c-88f9-4131-bfff-7a85f62930c8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:14.722051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fd9c449c-88f9-4131-bfff-7a85f62930c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:14.736525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-308c8532-0611-426c-9b37-ed68a714e332 None None] MFA Rules not processed for user `7e63b8e907764f2c818391ab2ad9184c`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:14.738519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-308c8532-0611-426c-9b37-ed68a714e332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:14.883890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 843/1687] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:55:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:14.903169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c8532-0611-426c-9b37-ed68a714e332 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:14.904765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-308c8532-0611-426c-9b37-ed68a714e332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:14.918880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 845/1688] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:14.921852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d714f0-8d11-4c5c-872e-d838522f7903 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:14.922360 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d714f0-8d11-4c5c-872e-d838522f7903 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:14.922823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d714f0-8d11-4c5c-872e-d838522f7903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:14.925407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308c8532-0611-426c-9b37-ed68a714e332 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:14.940873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733d719-4685-4f47-bc37-a887a383face service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:14.941568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733d719-4685-4f47-bc37-a887a383face service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:14.941873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733d719-4685-4f47-bc37-a887a383face service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:14.942161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733d719-4685-4f47-bc37-a887a383face service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:14.951817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733d719-4685-4f47-bc37-a887a383face service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:14.953472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733d719-4685-4f47-bc37-a887a383face service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:14.973436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 844/1689] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:55:14.974627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733d719-4685-4f47-bc37-a887a383face service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:14.976844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0733d719-4685-4f47-bc37-a887a383face service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:14.979550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d5d714f0-8d11-4c5c-872e-d838522f7903 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:14.980806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d5d714f0-8d11-4c5c-872e-d838522f7903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:15.005937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 846/1690] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:15.006989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d714f0-8d11-4c5c-872e-d838522f7903 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:15.020198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733d719-4685-4f47-bc37-a887a383face service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:15.029802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:15.034576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:15.039209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:15.039209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:15.039209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] PATH_INFO: `/v3/projects/c1b9ef2ffa8b458795820f28f4367791` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:15.039209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b9ef2ffa8b458795820f28f4367791)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:15.048561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:15.051218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d714f0-8d11-4c5c-872e-d838522f7903 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:15.074725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598d309-7faf-4751-8e4b-9099c1ceb4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:15.074725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598d309-7faf-4751-8e4b-9099c1ceb4fb admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:15.074725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598d309-7faf-4751-8e4b-9099c1ceb4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:15.074725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598d309-7faf-4751-8e4b-9099c1ceb4fb admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:15.075184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598d309-7faf-4751-8e4b-9099c1ceb4fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:55:15.075184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598d309-7faf-4751-8e4b-9099c1ceb4fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:15.075184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598d309-7faf-4751-8e4b-9099c1ceb4fb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:15.088584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:15.108515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 847/1691] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:55:15.114053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598d309-7faf-4751-8e4b-9099c1ceb4fb admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:15.147262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598d309-7faf-4751-8e4b-9099c1ceb4fb admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:15.164657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb3571-a2b0-49f8-9927-3c631c3207bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:15.166588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb3571-a2b0-49f8-9927-3c631c3207bb admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:15.166878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb3571-a2b0-49f8-9927-3c631c3207bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:15.167167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb3571-a2b0-49f8-9927-3c631c3207bb admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:15.170298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb3571-a2b0-49f8-9927-3c631c3207bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:15.172835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb3571-a2b0-49f8-9927-3c631c3207bb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:15.284852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b9ef2ffa8b458795820f28f4367791', 'request_id': 'req-371a97fb-a6bf-4564-9cdc-cb2330d31a69'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:15.301502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b9ef2ffa8b458795820f28f4367791', 'request_id': 'req-371a97fb-a6bf-4564-9cdc-cb2330d31a69'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:15.418096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b9ef2ffa8b458795820f28f4367791', 'request_id': 'req-371a97fb-a6bf-4564-9cdc-cb2330d31a69'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:15.440701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] The token cache is being invalidate because project c1b9ef2ffa8b458795820f28f4367791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:15.441093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:15.442108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-371a97fb-a6bf-4564-9cdc-cb2330d31a69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:15.445109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 845/1693] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:14 2022] DELETE /identity/v3/projects/c1b9ef2ffa8b458795820f28f4367791 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:15.445218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 848/1693] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:15 2022] POST /identity/v3/projects => generated 363 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:15.446637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb3571-a2b0-49f8-9927-3c631c3207bb admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:15.446761 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6327ba-4a43-4f07-ad37-2fe1603a11dc None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:15.446943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6327ba-4a43-4f07-ad37-2fe1603a11dc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:15.449444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6327ba-4a43-4f07-ad37-2fe1603a11dc None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:15.450317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-acfb3571-a2b0-49f8-9927-3c631c3207bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:15.454970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 846/1694] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jul 14 15:55:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:55:15.455679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6327ba-4a43-4f07-ad37-2fe1603a11dc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:15.464315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ce6327ba-4a43-4f07-ad37-2fe1603a11dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:15.627484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb3571-a2b0-49f8-9927-3c631c3207bb admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:15.641917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3441bb-edb6-41dd-9d80-098b20ec942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:15.642833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3441bb-edb6-41dd-9d80-098b20ec942e admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:15.643207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3441bb-edb6-41dd-9d80-098b20ec942e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:15.643558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3441bb-edb6-41dd-9d80-098b20ec942e admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:15.644403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3441bb-edb6-41dd-9d80-098b20ec942e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:15.648625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3441bb-edb6-41dd-9d80-098b20ec942e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:15.708766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6327ba-4a43-4f07-ad37-2fe1603a11dc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:15.722721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:15.723639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:15.723926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:15.724206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] PATH_INFO: `/v3/users/2a60cc08f2da45ee8241664fc8d868b9` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:15.730733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a60cc08f2da45ee8241664fc8d868b9)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:15.732725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:15.772306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 849/1695] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:15 2022] POST /identity/v3/users => generated 282 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:15.774804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12c494-9066-455c-a789-7629f1e91ebb None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:15.777085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12c494-9066-455c-a789-7629f1e91ebb None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:15.777810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12c494-9066-455c-a789-7629f1e91ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:15.835800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-bf12c494-9066-455c-a789-7629f1e91ebb None None] MFA Rules not processed for user `7e63b8e907764f2c818391ab2ad9184c`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:15.837098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-bf12c494-9066-455c-a789-7629f1e91ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:15.912223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 850/1696] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:55:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:15.913011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12c494-9066-455c-a789-7629f1e91ebb None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:15.927967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12c494-9066-455c-a789-7629f1e91ebb None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:15.942699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77b1cb-7e81-4f93-8de9-804e370cc9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:15.944982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77b1cb-7e81-4f93-8de9-804e370cc9de admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:15.944982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77b1cb-7e81-4f93-8de9-804e370cc9de admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:15.944982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77b1cb-7e81-4f93-8de9-804e370cc9de admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:15.945265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77b1cb-7e81-4f93-8de9-804e370cc9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:15.974798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77b1cb-7e81-4f93-8de9-804e370cc9de admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:16.008633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 851/1697] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:16.025972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e612b9-d8b4-4c21-930a-b3cc3febcf97 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:16.027755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e612b9-d8b4-4c21-930a-b3cc3febcf97 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:16.028090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e612b9-d8b4-4c21-930a-b3cc3febcf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:16.046346 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a60cc08f2da45ee8241664fc8d868b9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:16.095337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-39e612b9-d8b4-4c21-930a-b3cc3febcf97 None None] MFA Rules not processed for user `95c6d7130d6c4ae4b8460807e6820ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:16.096776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-39e612b9-d8b4-4c21-930a-b3cc3febcf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:16.132665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a60cc08f2da45ee8241664fc8d868b9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:16.133593 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a60cc08f2da45ee8241664fc8d868b9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:16.148513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a60cc08f2da45ee8241664fc8d868b9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:16.150225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a60cc08f2da45ee8241664fc8d868b9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:16.205435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 847/1698] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:15 2022] DELETE /identity/v3/users/2a60cc08f2da45ee8241664fc8d868b9 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:16.209253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:16.212503 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:16.251996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:16.302520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 852/1699] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:16.303260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e612b9-d8b4-4c21-930a-b3cc3febcf97 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:16.309071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-39e612b9-d8b4-4c21-930a-b3cc3febcf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:16.345682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b981f0f-efbe-4ecb-93ff-c9ae0fba878b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:16.349243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e612b9-d8b4-4c21-930a-b3cc3febcf97 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:16.357212 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:16.358117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:16.358448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:16.358743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:16.361248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:16.374374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:16.375905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:16.386411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 848/1700] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:16.387351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:16.389788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:16.529230 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037a99d-f709-4302-bf0b-e09bfabefa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:16.531048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037a99d-f709-4302-bf0b-e09bfabefa1a admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:16.531345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037a99d-f709-4302-bf0b-e09bfabefa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:16.531626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037a99d-f709-4302-bf0b-e09bfabefa1a admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:16.532414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037a99d-f709-4302-bf0b-e09bfabefa1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:16.535611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037a99d-f709-4302-bf0b-e09bfabefa1a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:16.561638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 853/1701] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:16.563072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbec6ec-5134-4a3c-b216-939ecbbbbe4e None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:16.563363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbec6ec-5134-4a3c-b216-939ecbbbbe4e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:16.563640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbec6ec-5134-4a3c-b216-939ecbbbbe4e None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:16.565051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 854/1702] 10.222.0.47 () {54 vars in 960 bytes} [Thu Jul 14 15:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:55:16.571378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbec6ec-5134-4a3c-b216-939ecbbbbe4e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:16.585492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4556a67-508e-40f8-969c-e28fa3dab467 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:16.598964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0caff5-cc23-454a-9ece-38d94987cc21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:16.600380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbec6ec-5134-4a3c-b216-939ecbbbbe4e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:16.602170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0caff5-cc23-454a-9ece-38d94987cc21 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:16.602559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0caff5-cc23-454a-9ece-38d94987cc21 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:16.606684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0caff5-cc23-454a-9ece-38d94987cc21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:16.607758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6f0caff5-cc23-454a-9ece-38d94987cc21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:16.614902 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1cb4f-b094-4cc1-a593-a816b95ab50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:16.618004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1cb4f-b094-4cc1-a593-a816b95ab50a admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:16.618313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1cb4f-b094-4cc1-a593-a816b95ab50a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:16.618600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1cb4f-b094-4cc1-a593-a816b95ab50a admin admin] PATH_INFO: `/v3/projects/a664a2b160004b819e161afdf3d96bd8/users/d9b9288d17b7454cadb1273c8fbb91ad/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:16.652785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1cb4f-b094-4cc1-a593-a816b95ab50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a664a2b160004b819e161afdf3d96bd8, user_id=d9b9288d17b7454cadb1273c8fbb91ad, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:16.659260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1cb4f-b094-4cc1-a593-a816b95ab50a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:16.714392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0caff5-cc23-454a-9ece-38d94987cc21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:16.718825 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0caff5-cc23-454a-9ece-38d94987cc21 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:16.727425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 849/1703] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:16.729074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dddb23-e12e-4175-b33a-5ee88a4eedbb None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:16.729074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dddb23-e12e-4175-b33a-5ee88a4eedbb None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:16.729344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dddb23-e12e-4175-b33a-5ee88a4eedbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:16.747455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 855/1704] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:16 2022] PUT /identity/v3/projects/a664a2b160004b819e161afdf3d96bd8/users/d9b9288d17b7454cadb1273c8fbb91ad/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:16.769125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-41dddb23-e12e-4175-b33a-5ee88a4eedbb None None] MFA Rules not processed for user `97524081e59f4775b83d82edf737810e`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:16.771730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-41dddb23-e12e-4175-b33a-5ee88a4eedbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:16.782108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c24d5-8179-423e-8750-aef3b6425505 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:16.782108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c24d5-8179-423e-8750-aef3b6425505 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:16.782108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c24d5-8179-423e-8750-aef3b6425505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:16.850424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-de1c24d5-8179-423e-8750-aef3b6425505 None None] MFA Rules not processed for user `d9b9288d17b7454cadb1273c8fbb91ad`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:16.853227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-de1c24d5-8179-423e-8750-aef3b6425505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:16.993398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 850/1705] 10.222.0.47 () {58 vars in 1057 bytes} [Thu Jul 14 15:55:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:17.005671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dddb23-e12e-4175-b33a-5ee88a4eedbb None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:17.033525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dddb23-e12e-4175-b33a-5ee88a4eedbb None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:17.046339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70f186-f038-4b21-998d-c112e45d031c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:17.047021 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70f186-f038-4b21-998d-c112e45d031c service placement] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:17.047321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70f186-f038-4b21-998d-c112e45d031c service placement] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:17.047595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70f186-f038-4b21-998d-c112e45d031c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:17.063191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9e70f186-f038-4b21-998d-c112e45d031c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:17.079415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70f186-f038-4b21-998d-c112e45d031c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:17.085653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70f186-f038-4b21-998d-c112e45d031c service placement] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:17.126659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 856/1706] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:17.138000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c24d5-8179-423e-8750-aef3b6425505 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:17.138517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 851/1707] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:17.162769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:17.446209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c24d5-8179-423e-8750-aef3b6425505 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:17.462484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02653c-723d-4bb4-9de7-0c9fbd118741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:17.463453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02653c-723d-4bb4-9de7-0c9fbd118741 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:17.463955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02653c-723d-4bb4-9de7-0c9fbd118741 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:17.474094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02653c-723d-4bb4-9de7-0c9fbd118741 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:17.474094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02653c-723d-4bb4-9de7-0c9fbd118741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:17.474094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02653c-723d-4bb4-9de7-0c9fbd118741 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:17.619866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 857/1708] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:17 2022] POST /identity/v3/projects => generated 365 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:17.621291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70f186-f038-4b21-998d-c112e45d031c service placement] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:17.641891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70f186-f038-4b21-998d-c112e45d031c service placement] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:17.677769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02653c-723d-4bb4-9de7-0c9fbd118741 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:17.681031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ce02653c-723d-4bb4-9de7-0c9fbd118741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:17.768294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:17.863557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60faa4d-158a-4a5c-98ea-3aafe6edbb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:17.875900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60faa4d-158a-4a5c-98ea-3aafe6edbb15 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:17.876345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60faa4d-158a-4a5c-98ea-3aafe6edbb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:17.876825 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60faa4d-158a-4a5c-98ea-3aafe6edbb15 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:17.878011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60faa4d-158a-4a5c-98ea-3aafe6edbb15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:17.901723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60faa4d-158a-4a5c-98ea-3aafe6edbb15 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:17.989412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02653c-723d-4bb4-9de7-0c9fbd118741 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:17.996308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:17.997590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:17.998081 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:17.999023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] PATH_INFO: `/v3/projects/f5d29d0ac9fc4ebf82c0d6907cbd689c` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:18.000613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5d29d0ac9fc4ebf82c0d6907cbd689c)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:18.003221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:18.089377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 852/1709] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:17 2022] POST /identity/v3/users => generated 289 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:18.089377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60faa4d-158a-4a5c-98ea-3aafe6edbb15 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:18.151773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60faa4d-158a-4a5c-98ea-3aafe6edbb15 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:18.194193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5d29d0ac9fc4ebf82c0d6907cbd689c', 'request_id': 'req-555b17a0-46dd-491b-aef2-71165319eeeb'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:18.212404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5d29d0ac9fc4ebf82c0d6907cbd689c', 'request_id': 'req-555b17a0-46dd-491b-aef2-71165319eeeb'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:18.332452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5d29d0ac9fc4ebf82c0d6907cbd689c', 'request_id': 'req-555b17a0-46dd-491b-aef2-71165319eeeb'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:18.354358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] The token cache is being invalidate because project f5d29d0ac9fc4ebf82c0d6907cbd689c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:18.354892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:18.355929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-555b17a0-46dd-491b-aef2-71165319eeeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:18.366277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 858/1710] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:17 2022] DELETE /identity/v3/projects/f5d29d0ac9fc4ebf82c0d6907cbd689c => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:55:18.370998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084898-cc78-4848-8f9e-11130b853bc8 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:18.372121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084898-cc78-4848-8f9e-11130b853bc8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:18.374642 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa084898-cc78-4848-8f9e-11130b853bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:18.445546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:18.452021 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c84db-5d47-4348-a35e-f3714b7f9b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:18.453168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c84db-5d47-4348-a35e-f3714b7f9b06 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:18.453668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c84db-5d47-4348-a35e-f3714b7f9b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:18.456921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c84db-5d47-4348-a35e-f3714b7f9b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:18.459073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c84db-5d47-4348-a35e-f3714b7f9b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:18.466492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-aa084898-cc78-4848-8f9e-11130b853bc8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:18.468203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c84db-5d47-4348-a35e-f3714b7f9b06 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:18.479607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-aa084898-cc78-4848-8f9e-11130b853bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:18.504593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 859/1711] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:18.516305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 853/1712] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:18.517418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f3bf7-3e71-4041-b600-a87f6e5eb6b9 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:18.517935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f3bf7-3e71-4041-b600-a87f6e5eb6b9 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:18.518980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f3bf7-3e71-4041-b600-a87f6e5eb6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:18.519412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c84db-5d47-4348-a35e-f3714b7f9b06 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:18.528401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ea3c84db-5d47-4348-a35e-f3714b7f9b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:18.567601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-048f3bf7-3e71-4041-b600-a87f6e5eb6b9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:18.570307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-048f3bf7-3e71-4041-b600-a87f6e5eb6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:18.589342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 860/1713] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:18.597323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f3bf7-3e71-4041-b600-a87f6e5eb6b9 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:18.606988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c84db-5d47-4348-a35e-f3714b7f9b06 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:18.620944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f3bf7-3e71-4041-b600-a87f6e5eb6b9 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:18.635838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd3e92-38c3-463c-917a-f7a9352f371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:18.637061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd3e92-38c3-463c-917a-f7a9352f371d admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:18.637527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd3e92-38c3-463c-917a-f7a9352f371d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:18.638165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd3e92-38c3-463c-917a-f7a9352f371d admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:18.639048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668de15-be9b-440f-b4bf-8cef4b0b1161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:18.641083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd3e92-38c3-463c-917a-f7a9352f371d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:55:18.642347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd3e92-38c3-463c-917a-f7a9352f371d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:18.647619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668de15-be9b-440f-b4bf-8cef4b0b1161 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:18.648530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd3e92-38c3-463c-917a-f7a9352f371d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:18.648885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668de15-be9b-440f-b4bf-8cef4b0b1161 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:18.648885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668de15-be9b-440f-b4bf-8cef4b0b1161 admin admin] PATH_INFO: `/v3/projects/834f0a72602441de9bc262f5b385d5a5/users/134fd6a2521344d893b9530ab21f363d/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:18.666737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 861/1714] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:55:18.675987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd3e92-38c3-463c-917a-f7a9352f371d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:18.706211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd3e92-38c3-463c-917a-f7a9352f371d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:18.718672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8487e3-f276-4531-8fdb-f1a285c3a7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:18.720695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8487e3-f276-4531-8fdb-f1a285c3a7fb admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:18.721036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8487e3-f276-4531-8fdb-f1a285c3a7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:18.721370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8487e3-f276-4531-8fdb-f1a285c3a7fb admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:18.723543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8487e3-f276-4531-8fdb-f1a285c3a7fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:18.727935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8487e3-f276-4531-8fdb-f1a285c3a7fb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:18.748158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668de15-be9b-440f-b4bf-8cef4b0b1161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834f0a72602441de9bc262f5b385d5a5, user_id=134fd6a2521344d893b9530ab21f363d, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:18.782573 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668de15-be9b-440f-b4bf-8cef4b0b1161 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:18.875488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 862/1715] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:18 2022] POST /identity/v3/projects => generated 361 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:18.881334 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8487e3-f276-4531-8fdb-f1a285c3a7fb admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:18.948414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 854/1716] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:18 2022] PUT /identity/v3/projects/834f0a72602441de9bc262f5b385d5a5/users/134fd6a2521344d893b9530ab21f363d/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:18.972771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668de15-be9b-440f-b4bf-8cef4b0b1161 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:19.123578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8487e3-f276-4531-8fdb-f1a285c3a7fb admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:19.180561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:19.213956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668de15-be9b-440f-b4bf-8cef4b0b1161 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:19.225151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee5849-01cf-4df6-8c43-77a561a148cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:19.227675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee5849-01cf-4df6-8c43-77a561a148cd admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:19.228497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee5849-01cf-4df6-8c43-77a561a148cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:19.229258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee5849-01cf-4df6-8c43-77a561a148cd admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:19.231862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee5849-01cf-4df6-8c43-77a561a148cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:19.238018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee5849-01cf-4df6-8c43-77a561a148cd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:19.252714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee597f-8dce-4215-902f-28225e0c011a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:19.254012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee597f-8dce-4215-902f-28225e0c011a admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:19.254265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee597f-8dce-4215-902f-28225e0c011a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:19.254979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee597f-8dce-4215-902f-28225e0c011a admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:19.256008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee597f-8dce-4215-902f-28225e0c011a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:19.260878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee597f-8dce-4215-902f-28225e0c011a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:19.272143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 855/1717] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:19.273378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee5849-01cf-4df6-8c43-77a561a148cd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:19.314394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee5849-01cf-4df6-8c43-77a561a148cd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:19.338848 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c96ae-227d-46ea-9cfd-3d85a7a1024a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:19.340804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c96ae-227d-46ea-9cfd-3d85a7a1024a admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:19.344529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c96ae-227d-46ea-9cfd-3d85a7a1024a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:19.345184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c96ae-227d-46ea-9cfd-3d85a7a1024a admin admin] PATH_INFO: `/v3/projects/834f0a72602441de9bc262f5b385d5a5/users/134fd6a2521344d893b9530ab21f363d/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:19.352037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c96ae-227d-46ea-9cfd-3d85a7a1024a admin admin] RBAC: Authorizing `identity:create_grant(project_id=834f0a72602441de9bc262f5b385d5a5, user_id=134fd6a2521344d893b9530ab21f363d, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:19.355746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c96ae-227d-46ea-9cfd-3d85a7a1024a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:19.370864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 856/1718] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:19 2022] PUT /identity/v3/projects/834f0a72602441de9bc262f5b385d5a5/users/134fd6a2521344d893b9530ab21f363d/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:19.372116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c96ae-227d-46ea-9cfd-3d85a7a1024a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:19.381007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-eb1c96ae-227d-46ea-9cfd-3d85a7a1024a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:19.419457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 863/1719] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:18 2022] POST /identity/v3/users => generated 281 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:19.420835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde13d7-d144-4651-bf57-65f2d3685fe2 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:19.424171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde13d7-d144-4651-bf57-65f2d3685fe2 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:19.424497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde13d7-d144-4651-bf57-65f2d3685fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:19.500214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:19.570842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-efde13d7-d144-4651-bf57-65f2d3685fe2 None None] MFA Rules not processed for user `134fd6a2521344d893b9530ab21f363d`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:19.573838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-efde13d7-d144-4651-bf57-65f2d3685fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:19.653139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c96ae-227d-46ea-9cfd-3d85a7a1024a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:19.659931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:19.673120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:19.676330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:19.677128 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:19.683148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:19.782757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 864/1720] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:19.783833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde13d7-d144-4651-bf57-65f2d3685fe2 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:19.886244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:19.887871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:19.936194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 857/1721] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:19.939143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:19.941755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:20.135036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde13d7-d144-4651-bf57-65f2d3685fe2 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:20.150133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2069b8f-cd57-408f-b107-f81e6adae0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:20.152112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2069b8f-cd57-408f-b107-f81e6adae0ac admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:20.152719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2069b8f-cd57-408f-b107-f81e6adae0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:20.153365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2069b8f-cd57-408f-b107-f81e6adae0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:20.154515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2069b8f-cd57-408f-b107-f81e6adae0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:20.156752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2069b8f-cd57-408f-b107-f81e6adae0ac admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:20.162421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e789f418-a0b2-4630-affa-9be241bbcd08 service nova] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:20.173143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da886216-e108-43fd-ac5c-1b7dc3adae52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:20.179400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 865/1722] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:20.179719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da886216-e108-43fd-ac5c-1b7dc3adae52 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:20.179719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da886216-e108-43fd-ac5c-1b7dc3adae52 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:20.179719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da886216-e108-43fd-ac5c-1b7dc3adae52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:20.179942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cda8ac-a815-4f0b-8d6d-a8e947318770 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:20.179942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cda8ac-a815-4f0b-8d6d-a8e947318770 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:20.179942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cda8ac-a815-4f0b-8d6d-a8e947318770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:20.187804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da886216-e108-43fd-ac5c-1b7dc3adae52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:20.189274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da886216-e108-43fd-ac5c-1b7dc3adae52 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:20.201776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-13cda8ac-a815-4f0b-8d6d-a8e947318770 None None] MFA Rules not processed for user `81ca45714ee44a549fd002b4aa13973b`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:20.207118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-13cda8ac-a815-4f0b-8d6d-a8e947318770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:20.212771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 858/1723] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:20.213688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da886216-e108-43fd-ac5c-1b7dc3adae52 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:20.254507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da886216-e108-43fd-ac5c-1b7dc3adae52 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:20.267502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822e76d-e7c5-42eb-a74d-fef8c95197ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:20.269982 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822e76d-e7c5-42eb-a74d-fef8c95197ad admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:20.270308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822e76d-e7c5-42eb-a74d-fef8c95197ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:20.273414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822e76d-e7c5-42eb-a74d-fef8c95197ad admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:20.273414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822e76d-e7c5-42eb-a74d-fef8c95197ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:20.278594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822e76d-e7c5-42eb-a74d-fef8c95197ad admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:20.294383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 859/1724] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:20.298658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822e76d-e7c5-42eb-a74d-fef8c95197ad admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:20.320932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822e76d-e7c5-42eb-a74d-fef8c95197ad admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:20.333113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da31eef-9fb4-4913-abc6-13e8bbb46bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:20.337486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da31eef-9fb4-4913-abc6-13e8bbb46bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:20.338627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da31eef-9fb4-4913-abc6-13e8bbb46bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:20.339542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da31eef-9fb4-4913-abc6-13e8bbb46bcd admin admin] PATH_INFO: `/v3/projects/4be1c18cc6314e069ded5660b5eda7ce/users/d6fccadc03ed4a558231a2407f6f4c8b/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:20.389630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:20.395681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da31eef-9fb4-4913-abc6-13e8bbb46bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4be1c18cc6314e069ded5660b5eda7ce, user_id=d6fccadc03ed4a558231a2407f6f4c8b, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:20.399299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da31eef-9fb4-4913-abc6-13e8bbb46bcd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:20.405225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 866/1725] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:20.541975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 860/1726] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:20 2022] PUT /identity/v3/projects/4be1c18cc6314e069ded5660b5eda7ce/users/d6fccadc03ed4a558231a2407f6f4c8b/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:20.555096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d80b8a-f16d-4da7-a396-26f11d59ae92 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:20.555628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d80b8a-f16d-4da7-a396-26f11d59ae92 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:20.556738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7fd81-de2c-4e96-a7d6-d7f2da6a9f05 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:20.557080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7fd81-de2c-4e96-a7d6-d7f2da6a9f05 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:20.557425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7fd81-de2c-4e96-a7d6-d7f2da6a9f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:20.558930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d80b8a-f16d-4da7-a396-26f11d59ae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:20.616865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-64d80b8a-f16d-4da7-a396-26f11d59ae92 None None] MFA Rules not processed for user `c4fd1155ff11417db8c647f0aadf74a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:20.619345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-64d80b8a-f16d-4da7-a396-26f11d59ae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:20.642056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-9bb7fd81-de2c-4e96-a7d6-d7f2da6a9f05 None None] MFA Rules not processed for user `d6fccadc03ed4a558231a2407f6f4c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:20.643325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9bb7fd81-de2c-4e96-a7d6-d7f2da6a9f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:20.795020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 867/1727] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:20.795668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d80b8a-f16d-4da7-a396-26f11d59ae92 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:20.883887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 861/1728] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:20.890435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7fd81-de2c-4e96-a7d6-d7f2da6a9f05 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:20.958320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d80b8a-f16d-4da7-a396-26f11d59ae92 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:20.988187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0560dd-e530-4e23-b39a-84b05514d4e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:20.988893 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0560dd-e530-4e23-b39a-84b05514d4e5 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:20.989346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0560dd-e530-4e23-b39a-84b05514d4e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:20.989859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0560dd-e530-4e23-b39a-84b05514d4e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:20.999224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7b0560dd-e530-4e23-b39a-84b05514d4e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:21.154923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7fd81-de2c-4e96-a7d6-d7f2da6a9f05 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:21.190648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b3e54-efd4-4f68-9524-69cb7d3cb69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:21.191501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b3e54-efd4-4f68-9524-69cb7d3cb69c admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:21.191796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b3e54-efd4-4f68-9524-69cb7d3cb69c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:21.192175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b3e54-efd4-4f68-9524-69cb7d3cb69c admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:21.192919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b3e54-efd4-4f68-9524-69cb7d3cb69c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:21.212899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b3e54-efd4-4f68-9524-69cb7d3cb69c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:21.245806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0560dd-e530-4e23-b39a-84b05514d4e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:21.250849 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0560dd-e530-4e23-b39a-84b05514d4e5 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:21.286635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 868/1729] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:20 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:21.356682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 862/1730] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:20 2022] POST /identity/v3/projects => generated 361 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:21.361131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0560dd-e530-4e23-b39a-84b05514d4e5 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:21.376970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0560dd-e530-4e23-b39a-84b05514d4e5 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:21.529561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:21.553327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce336610-e5d0-435c-9867-4276db930036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:21.554917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce336610-e5d0-435c-9867-4276db930036 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:21.555344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce336610-e5d0-435c-9867-4276db930036 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:21.555914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce336610-e5d0-435c-9867-4276db930036 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:21.558065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce336610-e5d0-435c-9867-4276db930036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:21.563021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce336610-e5d0-435c-9867-4276db930036 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:21.728443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b3e54-efd4-4f68-9524-69cb7d3cb69c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:21.730472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 869/1731] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:21 2022] POST /identity/v3/users => generated 287 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:21.748520 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b3e54-efd4-4f68-9524-69cb7d3cb69c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:21.757966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6e199-ac15-4af1-bc0a-51460d4a3af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:21.758969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6e199-ac15-4af1-bc0a-51460d4a3af4 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:21.759434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6e199-ac15-4af1-bc0a-51460d4a3af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:21.759856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e6e199-ac15-4af1-bc0a-51460d4a3af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:21.760749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6e199-ac15-4af1-bc0a-51460d4a3af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:21.762614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e6e199-ac15-4af1-bc0a-51460d4a3af4 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:21.785199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 863/1732] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:21.786291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce336610-e5d0-435c-9867-4276db930036 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:21.803676 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce336610-e5d0-435c-9867-4276db930036 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:21.820808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860abca2-4275-49a6-8ad9-d72013fea1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:21.820808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860abca2-4275-49a6-8ad9-d72013fea1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:21.820808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860abca2-4275-49a6-8ad9-d72013fea1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:21.822014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860abca2-4275-49a6-8ad9-d72013fea1ea admin admin] PATH_INFO: `/v3/projects/964a6edc4dd0488291b03db3f2df0a06/users/1ca48f3788dc4c7988ca8bc2e3e7d74c/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:21.865855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860abca2-4275-49a6-8ad9-d72013fea1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=964a6edc4dd0488291b03db3f2df0a06, user_id=1ca48f3788dc4c7988ca8bc2e3e7d74c, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:21.901409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860abca2-4275-49a6-8ad9-d72013fea1ea admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:22.073466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 870/1733] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:21 2022] PUT /identity/v3/projects/964a6edc4dd0488291b03db3f2df0a06/users/1ca48f3788dc4c7988ca8bc2e3e7d74c/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:22.078928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6e199-ac15-4af1-bc0a-51460d4a3af4 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:22.096651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e6e199-ac15-4af1-bc0a-51460d4a3af4 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:22.167789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:22.234664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9106dbd-2de5-4325-acbc-2c05278b2228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:22.235449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9106dbd-2de5-4325-acbc-2c05278b2228 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:22.235736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9106dbd-2de5-4325-acbc-2c05278b2228 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:22.236020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9106dbd-2de5-4325-acbc-2c05278b2228 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:22.237102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9106dbd-2de5-4325-acbc-2c05278b2228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:22.238981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9106dbd-2de5-4325-acbc-2c05278b2228 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:22.257954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 864/1734] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:22.262048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860abca2-4275-49a6-8ad9-d72013fea1ea admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:22.276446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860abca2-4275-49a6-8ad9-d72013fea1ea admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:22.282572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d556b-60ad-43ed-966f-63e39ceadca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:22.284171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d556b-60ad-43ed-966f-63e39ceadca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:22.284800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d556b-60ad-43ed-966f-63e39ceadca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:22.285221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d556b-60ad-43ed-966f-63e39ceadca2 admin admin] PATH_INFO: `/v3/projects/964a6edc4dd0488291b03db3f2df0a06/users/1ca48f3788dc4c7988ca8bc2e3e7d74c/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:22.287558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d556b-60ad-43ed-966f-63e39ceadca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=964a6edc4dd0488291b03db3f2df0a06, user_id=1ca48f3788dc4c7988ca8bc2e3e7d74c, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:22.291048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d556b-60ad-43ed-966f-63e39ceadca2 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:22.305869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 871/1735] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:22 2022] PUT /identity/v3/projects/964a6edc4dd0488291b03db3f2df0a06/users/1ca48f3788dc4c7988ca8bc2e3e7d74c/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:22.323418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7a7fc-c9b7-4af1-a58d-b2591f87d5a1 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:22.323418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7a7fc-c9b7-4af1-a58d-b2591f87d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:22.323418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7a7fc-c9b7-4af1-a58d-b2591f87d5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:22.409664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-1dc7a7fc-c9b7-4af1-a58d-b2591f87d5a1 None None] MFA Rules not processed for user `1ca48f3788dc4c7988ca8bc2e3e7d74c`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:22.415967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1dc7a7fc-c9b7-4af1-a58d-b2591f87d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:22.600915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 865/1736] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:22 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:22.612594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d556b-60ad-43ed-966f-63e39ceadca2 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:22.689111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d556b-60ad-43ed-966f-63e39ceadca2 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:22.694008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ac98b-5632-4624-95b4-0a72102d7a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:22.695320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ac98b-5632-4624-95b4-0a72102d7a0f service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:22.695612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ac98b-5632-4624-95b4-0a72102d7a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:22.696010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ac98b-5632-4624-95b4-0a72102d7a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:22.708688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ac98b-5632-4624-95b4-0a72102d7a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:22.710612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ac98b-5632-4624-95b4-0a72102d7a0f service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:22.723209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 872/1737] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:22 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:23.614292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c0e10-0c84-45b9-b684-2b98d9609af8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:23.614841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c0e10-0c84-45b9-b684-2b98d9609af8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:23.614841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c0e10-0c84-45b9-b684-2b98d9609af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:23.815118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-290c0e10-0c84-45b9-b684-2b98d9609af8 None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:23.817075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-290c0e10-0c84-45b9-b684-2b98d9609af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:24.108751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 866/1738] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:55:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:24.594349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:24.670732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ac98b-5632-4624-95b4-0a72102d7a0f service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:24.672470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c65ac98b-5632-4624-95b4-0a72102d7a0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:24.905426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c0e10-0c84-45b9-b684-2b98d9609af8 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:24.912547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ac98b-5632-4624-95b4-0a72102d7a0f service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:24.919905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417fd83-6c4e-4703-b2b6-ac4b72caa3bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:24.920897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417fd83-6c4e-4703-b2b6-ac4b72caa3bb service swift] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:24.921368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417fd83-6c4e-4703-b2b6-ac4b72caa3bb service swift] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:24.921879 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417fd83-6c4e-4703-b2b6-ac4b72caa3bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:24.935416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417fd83-6c4e-4703-b2b6-ac4b72caa3bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:24.941351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417fd83-6c4e-4703-b2b6-ac4b72caa3bb service swift] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:24.992175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 873/1739] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:55:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:55:25.032577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:25.215805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c0e10-0c84-45b9-b684-2b98d9609af8 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:25.237680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:25.238638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:25.240190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:25.240627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] PATH_INFO: `/v3/users/d6fccadc03ed4a558231a2407f6f4c8b` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:25.247686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6fccadc03ed4a558231a2407f6f4c8b)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:25.249662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:25.652815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:25.671953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6fccadc03ed4a558231a2407f6f4c8b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:25.745696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6fccadc03ed4a558231a2407f6f4c8b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:25.752922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6fccadc03ed4a558231a2407f6f4c8b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:25.766920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6fccadc03ed4a558231a2407f6f4c8b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:25.768220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6fccadc03ed4a558231a2407f6f4c8b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:25.868651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 867/1740] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:24 2022] DELETE /identity/v3/users/d6fccadc03ed4a558231a2407f6f4c8b => generated 0 bytes in 967 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:25.881637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53afa5-bf45-4a1d-b8aa-9a3320e7497e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:25.882526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53afa5-bf45-4a1d-b8aa-9a3320e7497e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:25.882793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53afa5-bf45-4a1d-b8aa-9a3320e7497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:25.952649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ea53afa5-bf45-4a1d-b8aa-9a3320e7497e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:25.976025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ea53afa5-bf45-4a1d-b8aa-9a3320e7497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:26.254897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 874/1741] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:26.344453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:26.356303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:26.559054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953dce8-c78a-4be2-a749-47be0e3475cd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:26.592920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e77bd-8792-47a2-9e76-89252f81c21d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:26.594238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e77bd-8792-47a2-9e76-89252f81c21d service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:26.594525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e77bd-8792-47a2-9e76-89252f81c21d service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:26.594816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e77bd-8792-47a2-9e76-89252f81c21d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:26.654229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e77bd-8792-47a2-9e76-89252f81c21d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:26.666658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e77bd-8792-47a2-9e76-89252f81c21d service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:26.709589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 868/1742] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:27.021446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d1a42-4f8f-4768-ba4f-7b1dc65e7f0e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:27.023384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d1a42-4f8f-4768-ba4f-7b1dc65e7f0e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:27.023873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d1a42-4f8f-4768-ba4f-7b1dc65e7f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:27.082909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-8b7d1a42-4f8f-4768-ba4f-7b1dc65e7f0e None None] MFA Rules not processed for user `7e63b8e907764f2c818391ab2ad9184c`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:27.084179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8b7d1a42-4f8f-4768-ba4f-7b1dc65e7f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:27.116636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 875/1743] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Jul 14 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:27.223598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e77bd-8792-47a2-9e76-89252f81c21d service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:27.225318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ac7e77bd-8792-47a2-9e76-89252f81c21d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:27.325381 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e77bd-8792-47a2-9e76-89252f81c21d service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:27.357982 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaf025-b383-4095-abc4-97b92ba109ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:27.358647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaf025-b383-4095-abc4-97b92ba109ae service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:27.358942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaf025-b383-4095-abc4-97b92ba109ae service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:27.359222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdaf025-b383-4095-abc4-97b92ba109ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:27.390088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdaf025-b383-4095-abc4-97b92ba109ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:27.394034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdaf025-b383-4095-abc4-97b92ba109ae service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:27.417097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 869/1744] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:27.673482 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1bc1b-7f8f-4ef3-a60b-ef6c4a7baa15 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:27.673770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1bc1b-7f8f-4ef3-a60b-ef6c4a7baa15 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:27.674237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1bc1b-7f8f-4ef3-a60b-ef6c4a7baa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:27.718555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-00d1bc1b-7f8f-4ef3-a60b-ef6c4a7baa15 None None] MFA Rules not processed for user `95c6d7130d6c4ae4b8460807e6820ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:27.720783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-00d1bc1b-7f8f-4ef3-a60b-ef6c4a7baa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:27.897083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 876/1745] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:28.015319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:29.599093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaf025-b383-4095-abc4-97b92ba109ae service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:29.601760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ebdaf025-b383-4095-abc4-97b92ba109ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:29.699354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdaf025-b383-4095-abc4-97b92ba109ae service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:29.715725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:29.718094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:29.718777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:29.719466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] PATH_INFO: `/v3/projects/4be1c18cc6314e069ded5660b5eda7ce` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:29.722575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4be1c18cc6314e069ded5660b5eda7ce)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:29.727592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:30.254540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:30.312991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4be1c18cc6314e069ded5660b5eda7ce', 'request_id': 'req-13b06125-1ce8-4462-b200-1dce15412ad5'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:30.350881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4be1c18cc6314e069ded5660b5eda7ce', 'request_id': 'req-13b06125-1ce8-4462-b200-1dce15412ad5'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:30.452411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4be1c18cc6314e069ded5660b5eda7ce', 'request_id': 'req-13b06125-1ce8-4462-b200-1dce15412ad5'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:30.482295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] The token cache is being invalidate because project 4be1c18cc6314e069ded5660b5eda7ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:30.482673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:30.487263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:30.489903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 870/1746] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:29 2022] DELETE /identity/v3/projects/4be1c18cc6314e069ded5660b5eda7ce => generated 0 bytes in 891 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:30.495134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1bc1b-7f8f-4ef3-a60b-ef6c4a7baa15 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:30.511851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1bc1b-7f8f-4ef3-a60b-ef6c4a7baa15 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:30.648559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:30.649461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:30.649751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:30.650550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] PATH_INFO: `/v3/users/1ca48f3788dc4c7988ca8bc2e3e7d74c` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:30.651445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca48f3788dc4c7988ca8bc2e3e7d74c)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:30.654220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:30.998244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca48f3788dc4c7988ca8bc2e3e7d74c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.118818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca48f3788dc4c7988ca8bc2e3e7d74c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.119448 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca48f3788dc4c7988ca8bc2e3e7d74c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.151008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca48f3788dc4c7988ca8bc2e3e7d74c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.151827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a418ad8d-0bb9-4fd4-aac1-ac04e7744e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca48f3788dc4c7988ca8bc2e3e7d74c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.175667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:31.177522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:31.217605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b06125-1ce8-4462-b200-1dce15412ad5 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:31.230916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 877/1747] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:30 2022] DELETE /identity/v3/users/1ca48f3788dc4c7988ca8bc2e3e7d74c => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:31.236845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b8e7e-909b-4654-ae92-7827b3bb2eb7 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:31.239401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b8e7e-909b-4654-ae92-7827b3bb2eb7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:31.240002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:31.240862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b8e7e-909b-4654-ae92-7827b3bb2eb7 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:31.244153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:31.245494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:31.248869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] PATH_INFO: `/v3/users/d9b9288d17b7454cadb1273c8fbb91ad` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:31.250212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 878/1748] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:55:31 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:55:31.253186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b9288d17b7454cadb1273c8fbb91ad)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:31.260390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:31.265110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185ea58-e21c-4303-9212-cb64263bb63e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:31.265743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185ea58-e21c-4303-9212-cb64263bb63e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:31.266334 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185ea58-e21c-4303-9212-cb64263bb63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:31.332186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6185ea58-e21c-4303-9212-cb64263bb63e None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:31.335925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6185ea58-e21c-4303-9212-cb64263bb63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:31.393016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:31.443531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 879/1749] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:55:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:31.582045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b9288d17b7454cadb1273c8fbb91ad'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.713728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b9288d17b7454cadb1273c8fbb91ad'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.714682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b9288d17b7454cadb1273c8fbb91ad'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.729292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b9288d17b7454cadb1273c8fbb91ad'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.730164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b9288d17b7454cadb1273c8fbb91ad'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:31.775519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 871/1750] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:31 2022] DELETE /identity/v3/users/d9b9288d17b7454cadb1273c8fbb91ad => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:55:31.777597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb9885-1f81-43e9-b6cb-4eb1798cdb6c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:31.778207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb9885-1f81-43e9-b6cb-4eb1798cdb6c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:31.778775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb9885-1f81-43e9-b6cb-4eb1798cdb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:31.820929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e3cb9885-1f81-43e9-b6cb-4eb1798cdb6c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:31.827948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e3cb9885-1f81-43e9-b6cb-4eb1798cdb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:31.997021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 880/1751] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:32.053895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:32.056916 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:32.300917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff31ecd-7bc7-4cc6-ac01-2d0f5973a345 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:32.309274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db85f57-96a0-4a4e-8fa7-00120ea9ed68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:32.309996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db85f57-96a0-4a4e-8fa7-00120ea9ed68 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:32.310490 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db85f57-96a0-4a4e-8fa7-00120ea9ed68 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:32.310936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db85f57-96a0-4a4e-8fa7-00120ea9ed68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:32.340301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db85f57-96a0-4a4e-8fa7-00120ea9ed68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:32.346729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db85f57-96a0-4a4e-8fa7-00120ea9ed68 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:32.367846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 872/1752] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:32.417991 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb9885-1f81-43e9-b6cb-4eb1798cdb6c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:32.419457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e3cb9885-1f81-43e9-b6cb-4eb1798cdb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:32.553423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb9885-1f81-43e9-b6cb-4eb1798cdb6c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:32.560493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021d6a9-d2c4-4a75-8545-9ee52d7a7146 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:32.560964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021d6a9-d2c4-4a75-8545-9ee52d7a7146 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:32.561271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021d6a9-d2c4-4a75-8545-9ee52d7a7146 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:32.561562 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021d6a9-d2c4-4a75-8545-9ee52d7a7146 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:32.563589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3021d6a9-d2c4-4a75-8545-9ee52d7a7146 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:32.578184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021d6a9-d2c4-4a75-8545-9ee52d7a7146 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:32.579869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021d6a9-d2c4-4a75-8545-9ee52d7a7146 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:32.612350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 881/1753] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:32.639130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db85f57-96a0-4a4e-8fa7-00120ea9ed68 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:32.640861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2db85f57-96a0-4a4e-8fa7-00120ea9ed68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:32.667180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db85f57-96a0-4a4e-8fa7-00120ea9ed68 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:32.689150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:32.690291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:32.690755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:32.691290 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] PATH_INFO: `/v3/projects/964a6edc4dd0488291b03db3f2df0a06` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:32.693376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=964a6edc4dd0488291b03db3f2df0a06)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:32.696452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:32.835435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:32.858445 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '964a6edc4dd0488291b03db3f2df0a06', 'request_id': 'req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:32.864685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '964a6edc4dd0488291b03db3f2df0a06', 'request_id': 'req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:32.937520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16117f-0093-4dbd-a551-d02a3cf5f9d2 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:32.938498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16117f-0093-4dbd-a551-d02a3cf5f9d2 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:32.938933 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16117f-0093-4dbd-a551-d02a3cf5f9d2 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:32.940868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 882/1754] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:55:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:55:32.956048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07149b9-3f44-44bb-bd55-60ed9008aeba None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:32.956710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07149b9-3f44-44bb-bd55-60ed9008aeba None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:32.957170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07149b9-3f44-44bb-bd55-60ed9008aeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:32.981288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-c07149b9-3f44-44bb-bd55-60ed9008aeba None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:32.982831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c07149b9-3f44-44bb-bd55-60ed9008aeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:32.989252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '964a6edc4dd0488291b03db3f2df0a06', 'request_id': 'req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:33.010351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] The token cache is being invalidate because project 964a6edc4dd0488291b03db3f2df0a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:33.010351 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:33.012435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-cb8dff93-89c3-4bd4-a93b-cdfa16c75850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:33.017906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 873/1755] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:32 2022] DELETE /identity/v3/projects/964a6edc4dd0488291b03db3f2df0a06 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:33.023065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fe46d-42c6-4d10-9c1f-0a5bbaf2cc00 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.023753 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fe46d-42c6-4d10-9c1f-0a5bbaf2cc00 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.024243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fe46d-42c6-4d10-9c1f-0a5bbaf2cc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.078866 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d13fe46d-42c6-4d10-9c1f-0a5bbaf2cc00 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:33.080140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d13fe46d-42c6-4d10-9c1f-0a5bbaf2cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:33.116850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 883/1756] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:55:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:33.135969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07149b9-3f44-44bb-bd55-60ed9008aeba None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:33.137428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c07149b9-3f44-44bb-bd55-60ed9008aeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:33.151835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07149b9-3f44-44bb-bd55-60ed9008aeba None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:33.159956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c38ead-b1ad-4a26-bd86-d31908e9e773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:33.160723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c38ead-b1ad-4a26-bd86-d31908e9e773 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.161759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c38ead-b1ad-4a26-bd86-d31908e9e773 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.162228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c38ead-b1ad-4a26-bd86-d31908e9e773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.175059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c38ead-b1ad-4a26-bd86-d31908e9e773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:33.177004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c38ead-b1ad-4a26-bd86-d31908e9e773 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:33.179033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 874/1757] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:33.189322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dddf1-271b-4d05-b568-330bc782d875 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.189611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dddf1-271b-4d05-b568-330bc782d875 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.190387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028dddf1-271b-4d05-b568-330bc782d875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.200686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 884/1758] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:55:33.212858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-028dddf1-271b-4d05-b568-330bc782d875 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:33.214115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-028dddf1-271b-4d05-b568-330bc782d875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:33.222199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 875/1759] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:33.226638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c38ead-b1ad-4a26-bd86-d31908e9e773 service nova] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:33.237286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c38ead-b1ad-4a26-bd86-d31908e9e773 service nova] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:33.242484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:33.242752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.243094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.243371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.244272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:55:33.244756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:33.246402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:33.259561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 885/1760] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:55:33.264545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dddf1-271b-4d05-b568-330bc782d875 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:33.281762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028dddf1-271b-4d05-b568-330bc782d875 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:33.292166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ef23d-e9bb-4f09-924a-7c66443aec27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:33.292166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ef23d-e9bb-4f09-924a-7c66443aec27 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.292166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ef23d-e9bb-4f09-924a-7c66443aec27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.292166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ef23d-e9bb-4f09-924a-7c66443aec27 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.292600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ef23d-e9bb-4f09-924a-7c66443aec27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:33.292600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ef23d-e9bb-4f09-924a-7c66443aec27 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:33.330361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:33.332543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:33.353167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca8a23-e582-46ff-ab18-afe61a04af8a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:33.358888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:33.360216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.360688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.361144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] PATH_INFO: `/v3/projects/a664a2b160004b819e161afdf3d96bd8` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.362471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a664a2b160004b819e161afdf3d96bd8)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:33.366983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:33.442130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 876/1761] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:33 2022] POST /identity/v3/projects => generated 369 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:33.445204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ef23d-e9bb-4f09-924a-7c66443aec27 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:33.534031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a664a2b160004b819e161afdf3d96bd8', 'request_id': 'req-0b16cd46-8229-4399-9433-81e4530fe48f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:33.559000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a664a2b160004b819e161afdf3d96bd8', 'request_id': 'req-0b16cd46-8229-4399-9433-81e4530fe48f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:33.575477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ef23d-e9bb-4f09-924a-7c66443aec27 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:33.619340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:33.679697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30202d1-27f0-48cb-ae03-470388f59a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:33.680977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30202d1-27f0-48cb-ae03-470388f59a05 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.681436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30202d1-27f0-48cb-ae03-470388f59a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.684071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30202d1-27f0-48cb-ae03-470388f59a05 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.684071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30202d1-27f0-48cb-ae03-470388f59a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:33.684380 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30202d1-27f0-48cb-ae03-470388f59a05 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:33.706711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a664a2b160004b819e161afdf3d96bd8', 'request_id': 'req-0b16cd46-8229-4399-9433-81e4530fe48f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:33.724550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] The token cache is being invalidate because project a664a2b160004b819e161afdf3d96bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:33.724785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:33.726070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:33.729867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 886/1762] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:33 2022] DELETE /identity/v3/projects/a664a2b160004b819e161afdf3d96bd8 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:55:33.732028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:33.733506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:33.750503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16cd46-8229-4399-9433-81e4530fe48f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:33.755887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:33.757067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.757184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.757403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] PATH_INFO: `/v3/users/134fd6a2521344d893b9530ab21f363d` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.758802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] RBAC: Authorizing `identity:delete_user(user_id=134fd6a2521344d893b9530ab21f363d)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:33.760855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:33.836077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 877/1763] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:33 2022] POST /identity/v3/users => generated 285 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:33.840962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30202d1-27f0-48cb-ae03-470388f59a05 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:33.843350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e30202d1-27f0-48cb-ae03-470388f59a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:33.858616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30202d1-27f0-48cb-ae03-470388f59a05 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:33.863845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33441497-d804-4104-a79c-6a39cf41efdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:33.864521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33441497-d804-4104-a79c-6a39cf41efdd admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.864800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33441497-d804-4104-a79c-6a39cf41efdd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.865077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33441497-d804-4104-a79c-6a39cf41efdd admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.865803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33441497-d804-4104-a79c-6a39cf41efdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:33.874148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33441497-d804-4104-a79c-6a39cf41efdd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:33.890084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 878/1764] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:55:33.901092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33441497-d804-4104-a79c-6a39cf41efdd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:33.918474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33441497-d804-4104-a79c-6a39cf41efdd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:33.924941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5d659-35d7-4ea0-a633-5aafbfa01a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:33.925642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5d659-35d7-4ea0-a633-5aafbfa01a17 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.926075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5d659-35d7-4ea0-a633-5aafbfa01a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.929289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5d659-35d7-4ea0-a633-5aafbfa01a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.929289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5d659-35d7-4ea0-a633-5aafbfa01a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:33.930125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5d659-35d7-4ea0-a633-5aafbfa01a17 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:33.944834 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 879/1765] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:55:33.949100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5d659-35d7-4ea0-a633-5aafbfa01a17 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:33.968472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5d659-35d7-4ea0-a633-5aafbfa01a17 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:33.972564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddaf06-4b81-43e6-b090-b5f5036da742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:33.973566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddaf06-4b81-43e6-b090-b5f5036da742 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:33.973963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddaf06-4b81-43e6-b090-b5f5036da742 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:33.974405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddaf06-4b81-43e6-b090-b5f5036da742 admin admin] PATH_INFO: `/v3/projects/7be3eaf9d0a3487c8e096de6d75af098/users/f509d25bbed44b469cc9d5f63b161199/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:33.994057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddaf06-4b81-43e6-b090-b5f5036da742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be3eaf9d0a3487c8e096de6d75af098, user_id=f509d25bbed44b469cc9d5f63b161199, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:33.997052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddaf06-4b81-43e6-b090-b5f5036da742 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:34.077149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '134fd6a2521344d893b9530ab21f363d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:34.092844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 880/1766] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:33 2022] PUT /identity/v3/projects/7be3eaf9d0a3487c8e096de6d75af098/users/f509d25bbed44b469cc9d5f63b161199/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:34.104908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c027b1-c22d-4be1-b208-61c0006d4f32 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:34.106034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c027b1-c22d-4be1-b208-61c0006d4f32 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:34.106960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c027b1-c22d-4be1-b208-61c0006d4f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:34.144483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e5c027b1-c22d-4be1-b208-61c0006d4f32 None None] MFA Rules not processed for user `f509d25bbed44b469cc9d5f63b161199`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:34.150810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e5c027b1-c22d-4be1-b208-61c0006d4f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:34.205531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '134fd6a2521344d893b9530ab21f363d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:34.206591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '134fd6a2521344d893b9530ab21f363d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:34.218006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '134fd6a2521344d893b9530ab21f363d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:34.218945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '134fd6a2521344d893b9530ab21f363d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:34.245531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 887/1767] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:33 2022] DELETE /identity/v3/users/134fd6a2521344d893b9530ab21f363d => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:34.260324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 881/1768] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:34.261538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:34.263682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:34.365786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076a1ce-b4ac-47da-83a3-bfd801d0d41d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:34.370668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb095b2-cd56-4544-acd7-d286dd43994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:34.372051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb095b2-cd56-4544-acd7-d286dd43994e admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:34.372051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb095b2-cd56-4544-acd7-d286dd43994e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:34.372051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb095b2-cd56-4544-acd7-d286dd43994e admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:34.372744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb095b2-cd56-4544-acd7-d286dd43994e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:34.374715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb095b2-cd56-4544-acd7-d286dd43994e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:34.469547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:34.478076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 888/1769] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:34 2022] POST /identity/v3/projects => generated 367 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:34.482023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c027b1-c22d-4be1-b208-61c0006d4f32 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:34.571761 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c027b1-c22d-4be1-b208-61c0006d4f32 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:34.660868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116ea87-837c-45c4-b2d5-9590f61f54fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:34.662711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116ea87-837c-45c4-b2d5-9590f61f54fe admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:34.663079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116ea87-837c-45c4-b2d5-9590f61f54fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:34.663450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116ea87-837c-45c4-b2d5-9590f61f54fe admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:34.664475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116ea87-837c-45c4-b2d5-9590f61f54fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:34.666500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2116ea87-837c-45c4-b2d5-9590f61f54fe admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:35.211790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 882/1770] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:34 2022] POST /identity/v3/users => generated 290 bytes in 730 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:35.215718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb095b2-cd56-4544-acd7-d286dd43994e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:35.226741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb095b2-cd56-4544-acd7-d286dd43994e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:35.232320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ae6a3-6abc-4c61-893b-8ace2b6eea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:35.233014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ae6a3-6abc-4c61-893b-8ace2b6eea86 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.233537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ae6a3-6abc-4c61-893b-8ace2b6eea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.234023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ae6a3-6abc-4c61-893b-8ace2b6eea86 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.234932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ae6a3-6abc-4c61-893b-8ace2b6eea86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:35.236745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ae6a3-6abc-4c61-893b-8ace2b6eea86 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:35.248451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 889/1771] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:35.254707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116ea87-837c-45c4-b2d5-9590f61f54fe admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:35.258940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ae6a3-6abc-4c61-893b-8ace2b6eea86 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:35.263658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-384ae6a3-6abc-4c61-893b-8ace2b6eea86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:35.267172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116ea87-837c-45c4-b2d5-9590f61f54fe admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:35.280606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192c299-5195-4f2d-99e7-475f14204444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:35.281689 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192c299-5195-4f2d-99e7-475f14204444 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.282072 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192c299-5195-4f2d-99e7-475f14204444 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.282433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192c299-5195-4f2d-99e7-475f14204444 admin admin] PATH_INFO: `/v3/projects/2250750c2a264630b390986cde17dc49/users/73189cedd04a4ef8a20f2619ead181ad/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.284018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ae6a3-6abc-4c61-893b-8ace2b6eea86 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:35.289608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:35.290613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.290977 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.291473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] PATH_INFO: `/v3/projects/834f0a72602441de9bc262f5b385d5a5` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.292925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834f0a72602441de9bc262f5b385d5a5)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:35.295557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:35.321371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192c299-5195-4f2d-99e7-475f14204444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2250750c2a264630b390986cde17dc49, user_id=73189cedd04a4ef8a20f2619ead181ad, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:35.329765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192c299-5195-4f2d-99e7-475f14204444 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:35.416787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 883/1772] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:35 2022] PUT /identity/v3/projects/2250750c2a264630b390986cde17dc49/users/73189cedd04a4ef8a20f2619ead181ad/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:35.427676 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192c299-5195-4f2d-99e7-475f14204444 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:35.507728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192c299-5195-4f2d-99e7-475f14204444 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:35.512440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c3e7a-421e-4016-9b97-e74d9fe13874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:35.513124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c3e7a-421e-4016-9b97-e74d9fe13874 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.513413 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c3e7a-421e-4016-9b97-e74d9fe13874 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.513723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c3e7a-421e-4016-9b97-e74d9fe13874 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.514489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c3e7a-421e-4016-9b97-e74d9fe13874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:35.516081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c3e7a-421e-4016-9b97-e74d9fe13874 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:35.528177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 884/1773] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:55:35.532375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c3e7a-421e-4016-9b97-e74d9fe13874 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:35.545483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c3e7a-421e-4016-9b97-e74d9fe13874 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:35.551245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305692b9-7830-4c87-834f-405f5994d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:35.552531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305692b9-7830-4c87-834f-405f5994d754 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.552814 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305692b9-7830-4c87-834f-405f5994d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.553183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305692b9-7830-4c87-834f-405f5994d754 admin admin] PATH_INFO: `/v3/projects/2250750c2a264630b390986cde17dc49/users/73189cedd04a4ef8a20f2619ead181ad/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.556343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305692b9-7830-4c87-834f-405f5994d754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2250750c2a264630b390986cde17dc49, user_id=73189cedd04a4ef8a20f2619ead181ad, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:35.559128 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305692b9-7830-4c87-834f-405f5994d754 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:35.573852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 885/1774] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:35 2022] PUT /identity/v3/projects/2250750c2a264630b390986cde17dc49/users/73189cedd04a4ef8a20f2619ead181ad/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:55:35.584656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834f0a72602441de9bc262f5b385d5a5', 'request_id': 'req-eb9a1726-79da-499e-a02d-8320d7ec0b34'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:35.590049 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebe097-e085-410c-92c3-ea2eb8f1ac41 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.590388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebe097-e085-410c-92c3-ea2eb8f1ac41 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.590671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebe097-e085-410c-92c3-ea2eb8f1ac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.590980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834f0a72602441de9bc262f5b385d5a5', 'request_id': 'req-eb9a1726-79da-499e-a02d-8320d7ec0b34'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:35.636483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-42ebe097-e085-410c-92c3-ea2eb8f1ac41 None None] MFA Rules not processed for user `73189cedd04a4ef8a20f2619ead181ad`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:35.637712 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-42ebe097-e085-410c-92c3-ea2eb8f1ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:35.674956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834f0a72602441de9bc262f5b385d5a5', 'request_id': 'req-eb9a1726-79da-499e-a02d-8320d7ec0b34'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:35.707283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] The token cache is being invalidate because project 834f0a72602441de9bc262f5b385d5a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:35.707739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:35.708526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-eb9a1726-79da-499e-a02d-8320d7ec0b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:35.710425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 890/1775] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:35 2022] DELETE /identity/v3/projects/834f0a72602441de9bc262f5b385d5a5 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:35.715468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354f997-ca4a-43b2-b66f-a9f65b874247 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.715820 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354f997-ca4a-43b2-b66f-a9f65b874247 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.716105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354f997-ca4a-43b2-b66f-a9f65b874247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.719083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 886/1776] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:35.733315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebe097-e085-410c-92c3-ea2eb8f1ac41 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:35.734776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-42ebe097-e085-410c-92c3-ea2eb8f1ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:35.744100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f354f997-ca4a-43b2-b66f-a9f65b874247 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:35.745564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f354f997-ca4a-43b2-b66f-a9f65b874247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:35.766662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:35.818794 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebe097-e085-410c-92c3-ea2eb8f1ac41 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:35.823057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90df198-ca7a-4b54-b6d0-376b4684d67f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:35.823957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90df198-ca7a-4b54-b6d0-376b4684d67f service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.824313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90df198-ca7a-4b54-b6d0-376b4684d67f service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.824595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90df198-ca7a-4b54-b6d0-376b4684d67f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.827294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e90df198-ca7a-4b54-b6d0-376b4684d67f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:35.845945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90df198-ca7a-4b54-b6d0-376b4684d67f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:35.846224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 891/1777] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:35.846348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90df198-ca7a-4b54-b6d0-376b4684d67f service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:35.851093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797fd9d-e0e1-4a86-8130-6714db69e1e6 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.853955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797fd9d-e0e1-4a86-8130-6714db69e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.853955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797fd9d-e0e1-4a86-8130-6714db69e1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.856404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 887/1778] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:55:35.879510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a797fd9d-e0e1-4a86-8130-6714db69e1e6 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:35.880902 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a797fd9d-e0e1-4a86-8130-6714db69e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:35.896973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 892/1779] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:35.902373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90df198-ca7a-4b54-b6d0-376b4684d67f service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:35.914116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90df198-ca7a-4b54-b6d0-376b4684d67f service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:35.918996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:35.919665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.919947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.920223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.920953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:55:35.921258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:35.922941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:35.941450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 888/1780] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:55:35.942055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797fd9d-e0e1-4a86-8130-6714db69e1e6 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:35.951873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797fd9d-e0e1-4a86-8130-6714db69e1e6 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:35.956969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757603c-8728-42be-8836-8899ce9ec3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:35.957800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757603c-8728-42be-8836-8899ce9ec3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:35.958143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757603c-8728-42be-8836-8899ce9ec3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:35.958423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757603c-8728-42be-8836-8899ce9ec3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:35.959258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757603c-8728-42be-8836-8899ce9ec3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:35.962002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1757603c-8728-42be-8836-8899ce9ec3d4 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:36.004546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:36.006665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:36.019510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692479b6-c150-4871-9d5a-a90a38aa7722 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:36.024059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af865efd-1bc2-4c69-81c7-ec657ce88de5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:36.024726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af865efd-1bc2-4c69-81c7-ec657ce88de5 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:36.025015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af865efd-1bc2-4c69-81c7-ec657ce88de5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:36.025350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af865efd-1bc2-4c69-81c7-ec657ce88de5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:36.032871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-af865efd-1bc2-4c69-81c7-ec657ce88de5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:36.128997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 893/1781] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:35 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:36.129230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af865efd-1bc2-4c69-81c7-ec657ce88de5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:36.130830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af865efd-1bc2-4c69-81c7-ec657ce88de5 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:36.134148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757603c-8728-42be-8836-8899ce9ec3d4 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:36.152175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 889/1782] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:55:36.239024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1757603c-8728-42be-8836-8899ce9ec3d4 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:36.249428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7444cd9-b601-4b5c-bb91-a4c2bba47b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:36.249428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7444cd9-b601-4b5c-bb91-a4c2bba47b63 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:36.249428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7444cd9-b601-4b5c-bb91-a4c2bba47b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:36.249428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7444cd9-b601-4b5c-bb91-a4c2bba47b63 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:36.249838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7444cd9-b601-4b5c-bb91-a4c2bba47b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:36.249838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7444cd9-b601-4b5c-bb91-a4c2bba47b63 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:36.397999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 894/1783] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:36 2022] POST /identity/v3/users => generated 294 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:36.399227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af865efd-1bc2-4c69-81c7-ec657ce88de5 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:36.419040 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af865efd-1bc2-4c69-81c7-ec657ce88de5 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:36.428987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b8db-0f91-4417-8734-9b762faeb335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:36.438519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b8db-0f91-4417-8734-9b762faeb335 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:36.438519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b8db-0f91-4417-8734-9b762faeb335 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:36.438519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b8db-0f91-4417-8734-9b762faeb335 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:36.438519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6b8db-0f91-4417-8734-9b762faeb335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:36.438519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6b8db-0f91-4417-8734-9b762faeb335 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:36.447633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 890/1784] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:36.448899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7444cd9-b601-4b5c-bb91-a4c2bba47b63 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:36.462742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7444cd9-b601-4b5c-bb91-a4c2bba47b63 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:36.468919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabb31c-ca88-4b37-818a-c9b01c4dc2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:36.470020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabb31c-ca88-4b37-818a-c9b01c4dc2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:36.470473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabb31c-ca88-4b37-818a-c9b01c4dc2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:36.470893 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabb31c-ca88-4b37-818a-c9b01c4dc2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:36.471800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabb31c-ca88-4b37-818a-c9b01c4dc2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:36.473649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabb31c-ca88-4b37-818a-c9b01c4dc2b6 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:36.493305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 895/1785] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:36.494514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b8db-0f91-4417-8734-9b762faeb335 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:36.505581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b8db-0f91-4417-8734-9b762faeb335 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:36.511010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de150-f0df-49f3-be8d-f0159f123694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:36.512276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de150-f0df-49f3-be8d-f0159f123694 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:36.512566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de150-f0df-49f3-be8d-f0159f123694 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:36.512841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de150-f0df-49f3-be8d-f0159f123694 admin admin] PATH_INFO: `/v3/projects/f41a2b1b5b954e17ad415536857bcd90/users/fb7fb2b78c65474581933e1a0d14c906/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:36.530077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1de150-f0df-49f3-be8d-f0159f123694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41a2b1b5b954e17ad415536857bcd90, user_id=fb7fb2b78c65474581933e1a0d14c906, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:36.532894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1de150-f0df-49f3-be8d-f0159f123694 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:36.609886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 891/1786] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:36 2022] PUT /identity/v3/projects/f41a2b1b5b954e17ad415536857bcd90/users/fb7fb2b78c65474581933e1a0d14c906/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:36.623890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7617cec-d779-4b15-8edf-0fff63d85a39 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:36.624256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7617cec-d779-4b15-8edf-0fff63d85a39 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:36.624623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7617cec-d779-4b15-8edf-0fff63d85a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:36.685902 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-b7617cec-d779-4b15-8edf-0fff63d85a39 None None] MFA Rules not processed for user `fb7fb2b78c65474581933e1a0d14c906`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:36.687159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b7617cec-d779-4b15-8edf-0fff63d85a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:36.763797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 896/1787] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:36.768377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de150-f0df-49f3-be8d-f0159f123694 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:36.834978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de150-f0df-49f3-be8d-f0159f123694 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:36.839818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717d2bf-6f97-43b6-b86b-8a2311d61b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:36.840574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717d2bf-6f97-43b6-b86b-8a2311d61b3e admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:36.840930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717d2bf-6f97-43b6-b86b-8a2311d61b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:36.841297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717d2bf-6f97-43b6-b86b-8a2311d61b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:36.842136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717d2bf-6f97-43b6-b86b-8a2311d61b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:36.844169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717d2bf-6f97-43b6-b86b-8a2311d61b3e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:36.995205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 892/1788] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:36 2022] POST /identity/v3/projects => generated 387 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:36.999214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7617cec-d779-4b15-8edf-0fff63d85a39 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:37.010416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7617cec-d779-4b15-8edf-0fff63d85a39 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:37.093766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8a7bb-1b56-42cc-86d7-71b5e080d55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:37.094502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8a7bb-1b56-42cc-86d7-71b5e080d55a admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:37.094791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8a7bb-1b56-42cc-86d7-71b5e080d55a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:37.095072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8a7bb-1b56-42cc-86d7-71b5e080d55a admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:37.095805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8a7bb-1b56-42cc-86d7-71b5e080d55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:37.097839 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8a7bb-1b56-42cc-86d7-71b5e080d55a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:37.103578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:37.220196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 897/1789] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:36 2022] POST /identity/v3/users => generated 300 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:37.224149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717d2bf-6f97-43b6-b86b-8a2311d61b3e admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:37.237342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717d2bf-6f97-43b6-b86b-8a2311d61b3e admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:37.244764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd00bf-1ec6-497e-adcd-f504ef6ca468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:37.245653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd00bf-1ec6-497e-adcd-f504ef6ca468 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:37.250584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd00bf-1ec6-497e-adcd-f504ef6ca468 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:37.250584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd00bf-1ec6-497e-adcd-f504ef6ca468 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:37.250584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd00bf-1ec6-497e-adcd-f504ef6ca468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:37.250584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd00bf-1ec6-497e-adcd-f504ef6ca468 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:37.266437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8a7bb-1b56-42cc-86d7-71b5e080d55a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:37.267623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 893/1790] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:55:37.281312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8a7bb-1b56-42cc-86d7-71b5e080d55a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:37.286564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59b047-4eea-44c6-a8df-55feae53decd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:37.287593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59b047-4eea-44c6-a8df-55feae53decd admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:37.288014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59b047-4eea-44c6-a8df-55feae53decd admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:37.288372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59b047-4eea-44c6-a8df-55feae53decd admin admin] PATH_INFO: `/v3/projects/76481d6501e84683b202739008ae28b9/users/be929b8e6ba44e4983eb9a16a5cd59ac/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:37.309424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59b047-4eea-44c6-a8df-55feae53decd admin admin] RBAC: Authorizing `identity:create_grant(project_id=76481d6501e84683b202739008ae28b9, user_id=be929b8e6ba44e4983eb9a16a5cd59ac, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:37.312538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59b047-4eea-44c6-a8df-55feae53decd admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:37.390151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 898/1791] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:37 2022] PUT /identity/v3/projects/76481d6501e84683b202739008ae28b9/users/be929b8e6ba44e4983eb9a16a5cd59ac/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:55:37.394387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd00bf-1ec6-497e-adcd-f504ef6ca468 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:37.414242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd00bf-1ec6-497e-adcd-f504ef6ca468 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:37.484921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f43da2-a4cb-4a12-877a-48c782e49292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:37.485958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f43da2-a4cb-4a12-877a-48c782e49292 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:37.486758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f43da2-a4cb-4a12-877a-48c782e49292 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:37.487327 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f43da2-a4cb-4a12-877a-48c782e49292 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:37.488210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f43da2-a4cb-4a12-877a-48c782e49292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:37.490221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f43da2-a4cb-4a12-877a-48c782e49292 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:37.504067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 894/1792] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:37.508476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59b047-4eea-44c6-a8df-55feae53decd admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:37.519999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59b047-4eea-44c6-a8df-55feae53decd admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:37.525567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf33c3-f9d8-4366-a1e9-7c8319d5c0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:37.526534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf33c3-f9d8-4366-a1e9-7c8319d5c0b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:37.526823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf33c3-f9d8-4366-a1e9-7c8319d5c0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:37.527100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf33c3-f9d8-4366-a1e9-7c8319d5c0b4 admin admin] PATH_INFO: `/v3/projects/76481d6501e84683b202739008ae28b9/users/be929b8e6ba44e4983eb9a16a5cd59ac/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:37.529184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf33c3-f9d8-4366-a1e9-7c8319d5c0b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76481d6501e84683b202739008ae28b9, user_id=be929b8e6ba44e4983eb9a16a5cd59ac, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:37.531917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf33c3-f9d8-4366-a1e9-7c8319d5c0b4 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:37.559041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 899/1793] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:37 2022] PUT /identity/v3/projects/76481d6501e84683b202739008ae28b9/users/be929b8e6ba44e4983eb9a16a5cd59ac/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:37.570007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13accd-a5a5-4ded-9da8-1b39d2aea616 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:37.570703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13accd-a5a5-4ded-9da8-1b39d2aea616 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:37.570703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13accd-a5a5-4ded-9da8-1b39d2aea616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:37.617880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cd13accd-a5a5-4ded-9da8-1b39d2aea616 None None] MFA Rules not processed for user `be929b8e6ba44e4983eb9a16a5cd59ac`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:37.627483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cd13accd-a5a5-4ded-9da8-1b39d2aea616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:37.668867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:37.747021 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 895/1794] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:37.769883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf33c3-f9d8-4366-a1e9-7c8319d5c0b4 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:37.769883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b1cf33c3-f9d8-4366-a1e9-7c8319d5c0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:37.847291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf33c3-f9d8-4366-a1e9-7c8319d5c0b4 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:37.849852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d52654-86e8-49da-9c15-eef7ba8e0208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:37.851181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52654-86e8-49da-9c15-eef7ba8e0208 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:37.851469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52654-86e8-49da-9c15-eef7ba8e0208 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:37.851754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d52654-86e8-49da-9c15-eef7ba8e0208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:37.862109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d52654-86e8-49da-9c15-eef7ba8e0208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:37.863688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d52654-86e8-49da-9c15-eef7ba8e0208 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:37.874322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 900/1795] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:38.182026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:38.246240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:39.114093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:39.428175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13accd-a5a5-4ded-9da8-1b39d2aea616 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:39.491730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13accd-a5a5-4ded-9da8-1b39d2aea616 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:39.496384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:39.497281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:39.497547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:39.497840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] PATH_INFO: `/v3/users/fb7fb2b78c65474581933e1a0d14c906` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:39.499066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb7fb2b78c65474581933e1a0d14c906)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:39.500990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:39.838907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb7fb2b78c65474581933e1a0d14c906'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:39.949332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb7fb2b78c65474581933e1a0d14c906'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:39.950002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb7fb2b78c65474581933e1a0d14c906'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:39.958969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb7fb2b78c65474581933e1a0d14c906'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:39.960217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb7fb2b78c65474581933e1a0d14c906'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:39.986387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 896/1796] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:39 2022] DELETE /identity/v3/users/fb7fb2b78c65474581933e1a0d14c906 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:39.991031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5a0b5-0c25-4d37-84a2-7830d500d379 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:39.991572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5a0b5-0c25-4d37-84a2-7830d500d379 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:39.991853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5a0b5-0c25-4d37-84a2-7830d500d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:40.014218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-60e5a0b5-0c25-4d37-84a2-7830d500d379 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:40.015535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-60e5a0b5-0c25-4d37-84a2-7830d500d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:40.100648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 901/1797] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:40.111568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:40.113895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:40.219765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:40.249252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76214db-24b0-454a-865f-0ea62898e222 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:40.267431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad578328-25ce-4461-af4c-1e0d31d508b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:40.268030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad578328-25ce-4461-af4c-1e0d31d508b6 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:40.268305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad578328-25ce-4461-af4c-1e0d31d508b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:40.270932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad578328-25ce-4461-af4c-1e0d31d508b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:40.286709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad578328-25ce-4461-af4c-1e0d31d508b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:40.288673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad578328-25ce-4461-af4c-1e0d31d508b6 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:40.300726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 897/1798] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:41.083508 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5a0b5-0c25-4d37-84a2-7830d500d379 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:41.086427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-60e5a0b5-0c25-4d37-84a2-7830d500d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:41.110308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5a0b5-0c25-4d37-84a2-7830d500d379 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:41.125104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:41.126155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:41.126651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:41.126929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] PATH_INFO: `/v3/projects/f41a2b1b5b954e17ad415536857bcd90` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:41.129220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41a2b1b5b954e17ad415536857bcd90)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:41.131176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:41.272886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:41.276540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41a2b1b5b954e17ad415536857bcd90', 'request_id': 'req-116a4670-5482-49d1-b4e4-251655e6bef0'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:41.284531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41a2b1b5b954e17ad415536857bcd90', 'request_id': 'req-116a4670-5482-49d1-b4e4-251655e6bef0'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:41.447294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41a2b1b5b954e17ad415536857bcd90', 'request_id': 'req-116a4670-5482-49d1-b4e4-251655e6bef0'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:41.464774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] The token cache is being invalidate because project f41a2b1b5b954e17ad415536857bcd90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:41.465283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:41.466141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:41.467716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 902/1799] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:41 2022] DELETE /identity/v3/projects/f41a2b1b5b954e17ad415536857bcd90 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:41.471871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad578328-25ce-4461-af4c-1e0d31d508b6 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:41.492026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad578328-25ce-4461-af4c-1e0d31d508b6 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:41.571899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:41.572792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:41.574426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:41.578511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] PATH_INFO: `/v3/users/be929b8e6ba44e4983eb9a16a5cd59ac` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:41.578511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be929b8e6ba44e4983eb9a16a5cd59ac)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:41.580215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:41.583737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:41.947577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be929b8e6ba44e4983eb9a16a5cd59ac'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.012466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be929b8e6ba44e4983eb9a16a5cd59ac'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.014414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be929b8e6ba44e4983eb9a16a5cd59ac'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.023383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be929b8e6ba44e4983eb9a16a5cd59ac'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.024108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be929b8e6ba44e4983eb9a16a5cd59ac'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.059452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 898/1800] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:41 2022] DELETE /identity/v3/users/be929b8e6ba44e4983eb9a16a5cd59ac => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:42.506320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:42.508033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:42.526727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a4670-5482-49d1-b4e4-251655e6bef0 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:42.615153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:42.617226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:42.617512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:42.617995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] PATH_INFO: `/v3/projects/76481d6501e84683b202739008ae28b9` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:42.619626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] RBAC: Authorizing `identity:delete_project(project_id=76481d6501e84683b202739008ae28b9)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:42.622891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:42.703226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:42.704785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:42.727303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3946363-7bcc-4e09-b32a-e9dc13947501 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:42.733340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:42.734584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:42.734874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:42.735147 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] PATH_INFO: `/v3/users/f509d25bbed44b469cc9d5f63b161199` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:42.737215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f509d25bbed44b469cc9d5f63b161199)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:42.740456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:42.791951 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76481d6501e84683b202739008ae28b9', 'request_id': 'req-e5837fee-690a-4d08-9dd7-acb14cbea26e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.799340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76481d6501e84683b202739008ae28b9', 'request_id': 'req-e5837fee-690a-4d08-9dd7-acb14cbea26e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.870758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76481d6501e84683b202739008ae28b9', 'request_id': 'req-e5837fee-690a-4d08-9dd7-acb14cbea26e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.888781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] The token cache is being invalidate because project 76481d6501e84683b202739008ae28b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:42.889066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.906823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e5837fee-690a-4d08-9dd7-acb14cbea26e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:42.915355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 903/1801] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:42 2022] DELETE /identity/v3/projects/76481d6501e84683b202739008ae28b9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:42.939075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7200aa-228c-47fc-a441-8f88314cf95d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:42.939535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7200aa-228c-47fc-a441-8f88314cf95d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:42.939855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7200aa-228c-47fc-a441-8f88314cf95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:42.986537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f509d25bbed44b469cc9d5f63b161199'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:43.010148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ce7200aa-228c-47fc-a441-8f88314cf95d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:43.011406 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ce7200aa-228c-47fc-a441-8f88314cf95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:43.085562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f509d25bbed44b469cc9d5f63b161199'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:43.086835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f509d25bbed44b469cc9d5f63b161199'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:43.098076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f509d25bbed44b469cc9d5f63b161199'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:43.100503 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-08b43d6f-1c70-4485-b868-8f3ef5a40c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f509d25bbed44b469cc9d5f63b161199'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:43.152891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 899/1802] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:42 2022] DELETE /identity/v3/users/f509d25bbed44b469cc9d5f63b161199 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:43.155998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b515d-d5cc-4ce4-bf74-54f35c061abf None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:43.156355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b515d-d5cc-4ce4-bf74-54f35c061abf None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:43.156646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b515d-d5cc-4ce4-bf74-54f35c061abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:43.163652 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 904/1803] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:43.167954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ccf73-4f46-41fe-bc9d-fd9a069409d0 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:43.168673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ccf73-4f46-41fe-bc9d-fd9a069409d0 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:43.169127 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ccf73-4f46-41fe-bc9d-fd9a069409d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:43.187545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:43.188495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-dc0b515d-d5cc-4ce4-bf74-54f35c061abf None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:43.191012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-dc0b515d-d5cc-4ce4-bf74-54f35c061abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:43.232297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a46ccf73-4f46-41fe-bc9d-fd9a069409d0 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:43.238350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a46ccf73-4f46-41fe-bc9d-fd9a069409d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:43.270144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:43.304906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 900/1804] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:43.324797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b515d-d5cc-4ce4-bf74-54f35c061abf None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:43.326347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-dc0b515d-d5cc-4ce4-bf74-54f35c061abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:43.416307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 905/1805] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:43.423786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ccf73-4f46-41fe-bc9d-fd9a069409d0 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:43.448340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ccf73-4f46-41fe-bc9d-fd9a069409d0 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:43.465231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b9188-49d3-4ffb-afc2-0831f88ed2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:43.466450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b9188-49d3-4ffb-afc2-0831f88ed2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:43.466911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b9188-49d3-4ffb-afc2-0831f88ed2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:43.470080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b9188-49d3-4ffb-afc2-0831f88ed2f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:43.470080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b9188-49d3-4ffb-afc2-0831f88ed2f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:55:43.470080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b9188-49d3-4ffb-afc2-0831f88ed2f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:43.473376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b9188-49d3-4ffb-afc2-0831f88ed2f1 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:43.488403 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b515d-d5cc-4ce4-bf74-54f35c061abf None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:43.493175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7b45d-fd6a-4027-81dc-d2f14953ef58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:43.493367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7b45d-fd6a-4027-81dc-d2f14953ef58 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:43.493547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7b45d-fd6a-4027-81dc-d2f14953ef58 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:43.493845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7b45d-fd6a-4027-81dc-d2f14953ef58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:43.500109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 906/1806] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:55:43.502462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b9188-49d3-4ffb-afc2-0831f88ed2f1 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:43.515009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7b45d-fd6a-4027-81dc-d2f14953ef58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:43.515216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b9188-49d3-4ffb-afc2-0831f88ed2f1 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:43.517398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7b45d-fd6a-4027-81dc-d2f14953ef58 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:43.521375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de09b4-66d2-4b61-8ecc-402c0df00604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:43.523131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de09b4-66d2-4b61-8ecc-402c0df00604 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:43.523679 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de09b4-66d2-4b61-8ecc-402c0df00604 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:43.524164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de09b4-66d2-4b61-8ecc-402c0df00604 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:43.525173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de09b4-66d2-4b61-8ecc-402c0df00604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:43.528555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de09b4-66d2-4b61-8ecc-402c0df00604 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:43.537144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 901/1807] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:43.629917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 907/1808] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:43 2022] POST /identity/v3/projects => generated 363 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:43.642715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7b45d-fd6a-4027-81dc-d2f14953ef58 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:43.663008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7b45d-fd6a-4027-81dc-d2f14953ef58 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:43.736402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667cd8-df9b-4c47-9dbc-477d1429a648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:43.737187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667cd8-df9b-4c47-9dbc-477d1429a648 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:43.737855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667cd8-df9b-4c47-9dbc-477d1429a648 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:43.737855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667cd8-df9b-4c47-9dbc-477d1429a648 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:43.738406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4667cd8-df9b-4c47-9dbc-477d1429a648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:43.741534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4667cd8-df9b-4c47-9dbc-477d1429a648 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:43.854901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 902/1809] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:43 2022] POST /identity/v3/users => generated 282 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:43.859044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de09b4-66d2-4b61-8ecc-402c0df00604 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:43.874809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de09b4-66d2-4b61-8ecc-402c0df00604 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:43.881200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb19c9-4ed5-42ec-a0ec-3ec09a19bcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:43.882574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb19c9-4ed5-42ec-a0ec-3ec09a19bcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:43.882574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb19c9-4ed5-42ec-a0ec-3ec09a19bcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:43.882574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb19c9-4ed5-42ec-a0ec-3ec09a19bcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:43.883539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb19c9-4ed5-42ec-a0ec-3ec09a19bcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:43.885659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb19c9-4ed5-42ec-a0ec-3ec09a19bcb5 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:43.897377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 908/1810] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:43.901718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667cd8-df9b-4c47-9dbc-477d1429a648 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:43.913701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667cd8-df9b-4c47-9dbc-477d1429a648 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:43.922843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533c123-a15e-4207-9820-a50bd5c264e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:43.923900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533c123-a15e-4207-9820-a50bd5c264e9 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:43.924410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533c123-a15e-4207-9820-a50bd5c264e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:43.924952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533c123-a15e-4207-9820-a50bd5c264e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:43.927015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533c123-a15e-4207-9820-a50bd5c264e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:43.930816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533c123-a15e-4207-9820-a50bd5c264e9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:43.946176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 903/1811] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:43.950679 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb19c9-4ed5-42ec-a0ec-3ec09a19bcb5 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:43.961669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb19c9-4ed5-42ec-a0ec-3ec09a19bcb5 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:43.967635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b10cfb-3f58-4139-a6ac-4833ad5477d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:43.968609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b10cfb-3f58-4139-a6ac-4833ad5477d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:43.968894 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b10cfb-3f58-4139-a6ac-4833ad5477d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:43.969134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b10cfb-3f58-4139-a6ac-4833ad5477d6 admin admin] PATH_INFO: `/v3/projects/c00b6dc54f564882b891b36f078b1c07/users/59d0d1585d3a4959a7434385a15989d7/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:43.989011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b10cfb-3f58-4139-a6ac-4833ad5477d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00b6dc54f564882b891b36f078b1c07, user_id=59d0d1585d3a4959a7434385a15989d7, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:43.992781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b10cfb-3f58-4139-a6ac-4833ad5477d6 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:44.076826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 909/1812] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:43 2022] PUT /identity/v3/projects/c00b6dc54f564882b891b36f078b1c07/users/59d0d1585d3a4959a7434385a15989d7/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:44.086545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10de365-9e66-4acd-9fee-a8a490952cab None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:44.087036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10de365-9e66-4acd-9fee-a8a490952cab None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:44.087348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10de365-9e66-4acd-9fee-a8a490952cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:44.116941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e10de365-9e66-4acd-9fee-a8a490952cab None None] MFA Rules not processed for user `59d0d1585d3a4959a7434385a15989d7`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:44.118223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e10de365-9e66-4acd-9fee-a8a490952cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:44.185443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 904/1813] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:44.198097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b10cfb-3f58-4139-a6ac-4833ad5477d6 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:44.200523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-36b10cfb-3f58-4139-a6ac-4833ad5477d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:44.214819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10de365-9e66-4acd-9fee-a8a490952cab None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:44.216279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e10de365-9e66-4acd-9fee-a8a490952cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:44.287234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b10cfb-3f58-4139-a6ac-4833ad5477d6 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:44.291959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b26f5-ee5e-4cd7-ab0e-e2763de390ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:44.292688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b26f5-ee5e-4cd7-ab0e-e2763de390ee service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:44.292976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b26f5-ee5e-4cd7-ab0e-e2763de390ee service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:44.293260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b26f5-ee5e-4cd7-ab0e-e2763de390ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:44.300763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10de365-9e66-4acd-9fee-a8a490952cab None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:44.305185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b26f5-ee5e-4cd7-ab0e-e2763de390ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:44.305185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b26f5-ee5e-4cd7-ab0e-e2763de390ee service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:44.305780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:44.306719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:44.307003 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:44.307496 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] PATH_INFO: `/v3/projects/7be3eaf9d0a3487c8e096de6d75af098` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:44.308464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be3eaf9d0a3487c8e096de6d75af098)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:44.310582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:44.316351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 910/1814] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:44.472985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be3eaf9d0a3487c8e096de6d75af098', 'request_id': 'req-2aa6c121-0257-4178-af64-09016b9f47aa'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:44.482439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be3eaf9d0a3487c8e096de6d75af098', 'request_id': 'req-2aa6c121-0257-4178-af64-09016b9f47aa'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:44.550114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be3eaf9d0a3487c8e096de6d75af098', 'request_id': 'req-2aa6c121-0257-4178-af64-09016b9f47aa'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:44.581308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] The token cache is being invalidate because project 7be3eaf9d0a3487c8e096de6d75af098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:44.584071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:44.591724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2aa6c121-0257-4178-af64-09016b9f47aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:44.594110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 905/1815] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:44 2022] DELETE /identity/v3/projects/7be3eaf9d0a3487c8e096de6d75af098 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:44.597979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b26f5-ee5e-4cd7-ab0e-e2763de390ee service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:44.615006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b26f5-ee5e-4cd7-ab0e-e2763de390ee service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:44.762421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:44.764894 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:44.765346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:44.765852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] PATH_INFO: `/v3/users/73189cedd04a4ef8a20f2619ead181ad` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:44.767613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73189cedd04a4ef8a20f2619ead181ad)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:44.769647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:44.832476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b4682-3410-462a-9375-70eebb8e5e0b None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:44.832923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b4682-3410-462a-9375-70eebb8e5e0b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:44.833113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b4682-3410-462a-9375-70eebb8e5e0b None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:44.835926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 906/1816] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:55:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:55:44.915202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377d9ed-dd37-41e1-8802-44e1c0a2f56f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:44.915870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377d9ed-dd37-41e1-8802-44e1c0a2f56f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:44.916158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377d9ed-dd37-41e1-8802-44e1c0a2f56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:44.959060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f377d9ed-dd37-41e1-8802-44e1c0a2f56f None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:44.959060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f377d9ed-dd37-41e1-8802-44e1c0a2f56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:45.006848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73189cedd04a4ef8a20f2619ead181ad'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:45.138536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73189cedd04a4ef8a20f2619ead181ad'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:45.139861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73189cedd04a4ef8a20f2619ead181ad'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:45.149559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73189cedd04a4ef8a20f2619ead181ad'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:45.151798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73189cedd04a4ef8a20f2619ead181ad'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:45.152800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 907/1817] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:55:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:45.185301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377d9ed-dd37-41e1-8802-44e1c0a2f56f None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:45.189497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f377d9ed-dd37-41e1-8802-44e1c0a2f56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:45.190714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 911/1818] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:44 2022] DELETE /identity/v3/users/73189cedd04a4ef8a20f2619ead181ad => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:45.236944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377d9ed-dd37-41e1-8802-44e1c0a2f56f None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:45.408664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474edec-07da-4274-8fad-8de5abc5317a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:45.409936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474edec-07da-4274-8fad-8de5abc5317a service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:45.410285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474edec-07da-4274-8fad-8de5abc5317a service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:45.410570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474edec-07da-4274-8fad-8de5abc5317a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:45.422945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474edec-07da-4274-8fad-8de5abc5317a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:45.425238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474edec-07da-4274-8fad-8de5abc5317a service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:45.440930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 908/1819] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:55:45.693802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:45.695290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:45.780840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad5a43-6d56-4b5c-9b22-5e4927308167 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:45.786644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:45.788327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:45.788782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:45.791362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] PATH_INFO: `/v3/projects/2250750c2a264630b390986cde17dc49` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:45.791362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2250750c2a264630b390986cde17dc49)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:45.794203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:45.982760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2250750c2a264630b390986cde17dc49', 'request_id': 'req-c83e157a-8d66-4a1b-b09e-31ba260557e8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:45.989015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2250750c2a264630b390986cde17dc49', 'request_id': 'req-c83e157a-8d66-4a1b-b09e-31ba260557e8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:46.187717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2250750c2a264630b390986cde17dc49', 'request_id': 'req-c83e157a-8d66-4a1b-b09e-31ba260557e8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:46.214322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] The token cache is being invalidate because project 2250750c2a264630b390986cde17dc49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:46.216531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:46.216999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c83e157a-8d66-4a1b-b09e-31ba260557e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:46.221356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 912/1820] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:45 2022] DELETE /identity/v3/projects/2250750c2a264630b390986cde17dc49 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:46.232437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a8ff3-d008-48bb-b2fd-0db8d033baea None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:46.232437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a8ff3-d008-48bb-b2fd-0db8d033baea None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:46.232437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a8ff3-d008-48bb-b2fd-0db8d033baea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:46.258238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-1f0a8ff3-d008-48bb-b2fd-0db8d033baea None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:46.259555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1f0a8ff3-d008-48bb-b2fd-0db8d033baea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:46.338845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 909/1821] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:46.346152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea250c5f-1677-4a49-a86b-b0ddf6b1c41c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:46.346636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea250c5f-1677-4a49-a86b-b0ddf6b1c41c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:46.347059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea250c5f-1677-4a49-a86b-b0ddf6b1c41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:46.394516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ea250c5f-1677-4a49-a86b-b0ddf6b1c41c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:46.396499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ea250c5f-1677-4a49-a86b-b0ddf6b1c41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:46.407323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 913/1822] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:46.411885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a8ff3-d008-48bb-b2fd-0db8d033baea None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:46.424251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a8ff3-d008-48bb-b2fd-0db8d033baea None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:46.433312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa4ecb-5ff4-4861-9205-37617a42e275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:46.433312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa4ecb-5ff4-4861-9205-37617a42e275 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:46.433312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa4ecb-5ff4-4861-9205-37617a42e275 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:46.433312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa4ecb-5ff4-4861-9205-37617a42e275 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:46.433621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa4ecb-5ff4-4861-9205-37617a42e275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:55:46.433621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa4ecb-5ff4-4861-9205-37617a42e275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:46.433788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa4ecb-5ff4-4861-9205-37617a42e275 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:46.454718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 910/1823] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:55:46.461334 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea250c5f-1677-4a49-a86b-b0ddf6b1c41c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:46.486266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea250c5f-1677-4a49-a86b-b0ddf6b1c41c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:46.511944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032e189-1b5a-41a2-ad07-a3184991399a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:46.512648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032e189-1b5a-41a2-ad07-a3184991399a admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:46.512970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032e189-1b5a-41a2-ad07-a3184991399a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:46.513218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032e189-1b5a-41a2-ad07-a3184991399a admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:46.514160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032e189-1b5a-41a2-ad07-a3184991399a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:46.516321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1032e189-1b5a-41a2-ad07-a3184991399a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:46.633528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 914/1824] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:46 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:46.637901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa4ecb-5ff4-4861-9205-37617a42e275 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:46.654094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa4ecb-5ff4-4861-9205-37617a42e275 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:46.794084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14158a-fa74-4a6f-9251-a4a140907656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:46.795399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14158a-fa74-4a6f-9251-a4a140907656 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:46.796683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14158a-fa74-4a6f-9251-a4a140907656 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:46.800364 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14158a-fa74-4a6f-9251-a4a140907656 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:46.800364 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14158a-fa74-4a6f-9251-a4a140907656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:46.802318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14158a-fa74-4a6f-9251-a4a140907656 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:47.019295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:47.060892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 911/1825] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:46 2022] POST /identity/v3/users => generated 293 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:47.069303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032e189-1b5a-41a2-ad07-a3184991399a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:47.088127 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032e189-1b5a-41a2-ad07-a3184991399a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:47.103019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16217623-b832-4824-aa90-85ef292fa108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:47.103874 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16217623-b832-4824-aa90-85ef292fa108 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:47.104270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16217623-b832-4824-aa90-85ef292fa108 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:47.105232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16217623-b832-4824-aa90-85ef292fa108 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:47.106753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16217623-b832-4824-aa90-85ef292fa108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:47.108598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16217623-b832-4824-aa90-85ef292fa108 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:47.115020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:47.121719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 915/1826] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:47.126844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14158a-fa74-4a6f-9251-a4a140907656 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:47.138046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14158a-fa74-4a6f-9251-a4a140907656 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:47.143672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790082e9-b222-46a3-b53c-5f71a5826c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:47.144667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790082e9-b222-46a3-b53c-5f71a5826c2e admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:47.144989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790082e9-b222-46a3-b53c-5f71a5826c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:47.145283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790082e9-b222-46a3-b53c-5f71a5826c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:47.151714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790082e9-b222-46a3-b53c-5f71a5826c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:47.151714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790082e9-b222-46a3-b53c-5f71a5826c2e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:47.162481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 912/1827] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:47.165643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16217623-b832-4824-aa90-85ef292fa108 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:47.191387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16217623-b832-4824-aa90-85ef292fa108 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:47.198145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ca257-d996-4b00-a259-16c7fb1c60d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:47.199419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ca257-d996-4b00-a259-16c7fb1c60d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:47.199854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ca257-d996-4b00-a259-16c7fb1c60d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:47.200331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ca257-d996-4b00-a259-16c7fb1c60d9 admin admin] PATH_INFO: `/v3/projects/f0b84ea38ce4408aa417b5593728e745/users/2d5c1ff0f9094c5a9cad57ca8413f38e/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:47.219827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ca257-d996-4b00-a259-16c7fb1c60d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b84ea38ce4408aa417b5593728e745, user_id=2d5c1ff0f9094c5a9cad57ca8413f38e, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:47.223123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ca257-d996-4b00-a259-16c7fb1c60d9 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:47.337509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 916/1828] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:47 2022] PUT /identity/v3/projects/f0b84ea38ce4408aa417b5593728e745/users/2d5c1ff0f9094c5a9cad57ca8413f38e/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:47.340590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc94a67-ae6d-4455-996b-7d1ebc717899 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:47.350894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc94a67-ae6d-4455-996b-7d1ebc717899 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:47.350894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc94a67-ae6d-4455-996b-7d1ebc717899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:47.383157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-1fc94a67-ae6d-4455-996b-7d1ebc717899 None None] MFA Rules not processed for user `2d5c1ff0f9094c5a9cad57ca8413f38e`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:47.385015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1fc94a67-ae6d-4455-996b-7d1ebc717899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:47.494678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 913/1829] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:47.499306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ca257-d996-4b00-a259-16c7fb1c60d9 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:47.661122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ca257-d996-4b00-a259-16c7fb1c60d9 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:47.702422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697bba1-cb5b-4348-a3d6-818d20441eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:47.703861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697bba1-cb5b-4348-a3d6-818d20441eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:47.704344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697bba1-cb5b-4348-a3d6-818d20441eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:47.704692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697bba1-cb5b-4348-a3d6-818d20441eb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:47.705544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697bba1-cb5b-4348-a3d6-818d20441eb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:47.707737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697bba1-cb5b-4348-a3d6-818d20441eb1 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:47.823298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 917/1830] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:47 2022] POST /identity/v3/projects => generated 385 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:47.824352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc94a67-ae6d-4455-996b-7d1ebc717899 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:47.835860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc94a67-ae6d-4455-996b-7d1ebc717899 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:47.895427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b3704-87d3-48df-b8ea-c3588a1bc0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:47.896239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b3704-87d3-48df-b8ea-c3588a1bc0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:47.896519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b3704-87d3-48df-b8ea-c3588a1bc0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:47.896815 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b3704-87d3-48df-b8ea-c3588a1bc0d3 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:47.898064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b3704-87d3-48df-b8ea-c3588a1bc0d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:47.900020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b3704-87d3-48df-b8ea-c3588a1bc0d3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:48.018762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 914/1831] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:47 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:48.019984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697bba1-cb5b-4348-a3d6-818d20441eb1 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:48.030651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697bba1-cb5b-4348-a3d6-818d20441eb1 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:48.035208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc527a73-8b11-4edb-a9d1-05dde5940092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:48.035890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc527a73-8b11-4edb-a9d1-05dde5940092 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:48.036401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc527a73-8b11-4edb-a9d1-05dde5940092 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:48.036661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc527a73-8b11-4edb-a9d1-05dde5940092 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:48.037459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc527a73-8b11-4edb-a9d1-05dde5940092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:48.045837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc527a73-8b11-4edb-a9d1-05dde5940092 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:48.056464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 918/1832] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:48.060813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b3704-87d3-48df-b8ea-c3588a1bc0d3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:48.070776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b3704-87d3-48df-b8ea-c3588a1bc0d3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:48.075524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31d04-8fd1-425d-96b5-5b6ea693013c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:48.076136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31d04-8fd1-425d-96b5-5b6ea693013c admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:48.076415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31d04-8fd1-425d-96b5-5b6ea693013c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:48.076692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31d04-8fd1-425d-96b5-5b6ea693013c admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:48.077418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a31d04-8fd1-425d-96b5-5b6ea693013c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:48.079097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a31d04-8fd1-425d-96b5-5b6ea693013c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:48.090106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 915/1833] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:55:48.094207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc527a73-8b11-4edb-a9d1-05dde5940092 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:48.105212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc527a73-8b11-4edb-a9d1-05dde5940092 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:48.110365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cfef5-de54-48df-9bd7-04cd60d6abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:48.112043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cfef5-de54-48df-9bd7-04cd60d6abdb admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:48.112336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cfef5-de54-48df-9bd7-04cd60d6abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:48.112789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cfef5-de54-48df-9bd7-04cd60d6abdb admin admin] PATH_INFO: `/v3/projects/8a913be2eaaa4a94b84c315b8d61db02/users/95debba9747f498a90bed24a0273d773/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:48.131001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cfef5-de54-48df-9bd7-04cd60d6abdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a913be2eaaa4a94b84c315b8d61db02, user_id=95debba9747f498a90bed24a0273d773, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:48.134482 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cfef5-de54-48df-9bd7-04cd60d6abdb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:48.201735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 919/1834] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:48 2022] PUT /identity/v3/projects/8a913be2eaaa4a94b84c315b8d61db02/users/95debba9747f498a90bed24a0273d773/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:48.212877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf5e32-1e39-4013-b013-8a78f7bff3e8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:48.213188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf5e32-1e39-4013-b013-8a78f7bff3e8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:48.213518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf5e32-1e39-4013-b013-8a78f7bff3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:48.241550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f6cf5e32-1e39-4013-b013-8a78f7bff3e8 None None] MFA Rules not processed for user `95debba9747f498a90bed24a0273d773`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:48.243038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f6cf5e32-1e39-4013-b013-8a78f7bff3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:48.317855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 916/1835] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:48.321259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cfef5-de54-48df-9bd7-04cd60d6abdb admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:48.402975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cfef5-de54-48df-9bd7-04cd60d6abdb admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:48.403544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7c4fe-860c-4ae4-97ce-4adfe3974456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:48.404234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7c4fe-860c-4ae4-97ce-4adfe3974456 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:48.404520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7c4fe-860c-4ae4-97ce-4adfe3974456 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:48.404801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7c4fe-860c-4ae4-97ce-4adfe3974456 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:48.405505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7c4fe-860c-4ae4-97ce-4adfe3974456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:48.407663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7c4fe-860c-4ae4-97ce-4adfe3974456 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:48.556332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 920/1836] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:48 2022] POST /identity/v3/projects => generated 387 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:48.560709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf5e32-1e39-4013-b013-8a78f7bff3e8 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:48.574461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf5e32-1e39-4013-b013-8a78f7bff3e8 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:48.685233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bce9b9-35d4-4099-b3a0-24ec93ccd266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:48.686057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bce9b9-35d4-4099-b3a0-24ec93ccd266 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:48.686451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bce9b9-35d4-4099-b3a0-24ec93ccd266 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:48.686802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bce9b9-35d4-4099-b3a0-24ec93ccd266 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:48.687594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bce9b9-35d4-4099-b3a0-24ec93ccd266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:48.689680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bce9b9-35d4-4099-b3a0-24ec93ccd266 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:49.212914 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 917/1837] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:48 2022] POST /identity/v3/users => generated 300 bytes in 653 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:49.220324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7c4fe-860c-4ae4-97ce-4adfe3974456 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:49.228169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7c4fe-860c-4ae4-97ce-4adfe3974456 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:49.232724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2dd7f-050d-450d-8411-0df261a6a075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:49.233395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2dd7f-050d-450d-8411-0df261a6a075 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:49.233726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2dd7f-050d-450d-8411-0df261a6a075 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:49.234030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2dd7f-050d-450d-8411-0df261a6a075 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:49.235298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2dd7f-050d-450d-8411-0df261a6a075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:49.236389 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2dd7f-050d-450d-8411-0df261a6a075 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:49.248421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 921/1838] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:49.252461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bce9b9-35d4-4099-b3a0-24ec93ccd266 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:49.262700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bce9b9-35d4-4099-b3a0-24ec93ccd266 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:49.267282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18754d88-b828-4913-93a9-2cbd031bfaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:49.268174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18754d88-b828-4913-93a9-2cbd031bfaf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:49.268460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18754d88-b828-4913-93a9-2cbd031bfaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:49.268734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18754d88-b828-4913-93a9-2cbd031bfaf1 admin admin] PATH_INFO: `/v3/projects/fa2bedfd90f8423785a23358e3a404b5/users/1efab6f85f2847c7a8ee4dff26b5c009/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:49.285951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18754d88-b828-4913-93a9-2cbd031bfaf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2bedfd90f8423785a23358e3a404b5, user_id=1efab6f85f2847c7a8ee4dff26b5c009, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:49.289487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18754d88-b828-4913-93a9-2cbd031bfaf1 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:49.377583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 918/1839] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:49 2022] PUT /identity/v3/projects/fa2bedfd90f8423785a23358e3a404b5/users/1efab6f85f2847c7a8ee4dff26b5c009/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:49.389064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2dd7f-050d-450d-8411-0df261a6a075 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:49.391675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2dd7f-050d-450d-8411-0df261a6a075 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:49.449550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879e575-bbb3-4090-ad33-ce25bb40de46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:49.457776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e575-bbb3-4090-ad33-ce25bb40de46 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:49.457776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e575-bbb3-4090-ad33-ce25bb40de46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:49.457776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879e575-bbb3-4090-ad33-ce25bb40de46 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:49.457776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879e575-bbb3-4090-ad33-ce25bb40de46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:49.457776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879e575-bbb3-4090-ad33-ce25bb40de46 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:49.463718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 922/1840] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:49.468088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18754d88-b828-4913-93a9-2cbd031bfaf1 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:49.478811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18754d88-b828-4913-93a9-2cbd031bfaf1 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:49.483389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c07b2-b4d9-4137-afaf-025fc97c6911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:49.484278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c07b2-b4d9-4137-afaf-025fc97c6911 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:49.484560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c07b2-b4d9-4137-afaf-025fc97c6911 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:49.484833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c07b2-b4d9-4137-afaf-025fc97c6911 admin admin] PATH_INFO: `/v3/projects/fa2bedfd90f8423785a23358e3a404b5/users/1efab6f85f2847c7a8ee4dff26b5c009/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:49.486816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c07b2-b4d9-4137-afaf-025fc97c6911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2bedfd90f8423785a23358e3a404b5, user_id=1efab6f85f2847c7a8ee4dff26b5c009, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:49.489423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c07b2-b4d9-4137-afaf-025fc97c6911 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:49.499476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 919/1841] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:49 2022] PUT /identity/v3/projects/fa2bedfd90f8423785a23358e3a404b5/users/1efab6f85f2847c7a8ee4dff26b5c009/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:49.507140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:49.510634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8c2ab-0e6f-46c2-9541-dc1bcfa304d3 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:49.510926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8c2ab-0e6f-46c2-9541-dc1bcfa304d3 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:49.511206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8c2ab-0e6f-46c2-9541-dc1bcfa304d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:49.538476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-03a8c2ab-0e6f-46c2-9541-dc1bcfa304d3 None None] MFA Rules not processed for user `1efab6f85f2847c7a8ee4dff26b5c009`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:49.539722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-03a8c2ab-0e6f-46c2-9541-dc1bcfa304d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:49.543145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:49.607615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 923/1842] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:49.618474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c07b2-b4d9-4137-afaf-025fc97c6911 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:49.620316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b40c07b2-b4d9-4137-afaf-025fc97c6911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:49.677006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c07b2-b4d9-4137-afaf-025fc97c6911 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:49.680568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2be853-0d53-4351-88b4-2158d8845533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:49.681244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2be853-0d53-4351-88b4-2158d8845533 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:49.681527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2be853-0d53-4351-88b4-2158d8845533 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:49.681803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2be853-0d53-4351-88b4-2158d8845533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:49.694227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2be853-0d53-4351-88b4-2158d8845533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:49.694227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2be853-0d53-4351-88b4-2158d8845533 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:49.704264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 920/1843] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:51.019250 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8c2ab-0e6f-46c2-9541-dc1bcfa304d3 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:51.022474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-03a8c2ab-0e6f-46c2-9541-dc1bcfa304d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:51.052644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8c2ab-0e6f-46c2-9541-dc1bcfa304d3 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:51.057712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1af7b-cc54-4191-8457-cd7f0478055a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:51.058439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1af7b-cc54-4191-8457-cd7f0478055a service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:51.058885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1af7b-cc54-4191-8457-cd7f0478055a service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:51.059167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1af7b-cc54-4191-8457-cd7f0478055a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:51.153720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1af7b-cc54-4191-8457-cd7f0478055a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:51.155563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1af7b-cc54-4191-8457-cd7f0478055a service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:51.167475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 924/1844] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:51.651729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:51.720763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:53.206574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:53.733499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:55.661461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2be853-0d53-4351-88b4-2158d8845533 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:55.664256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6e2be853-0d53-4351-88b4-2158d8845533 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:55.752188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2be853-0d53-4351-88b4-2158d8845533 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:55.762526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:55.763841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:55.764874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:55.765468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] PATH_INFO: `/v3/users/876156edcf9643beaf74738d31177041` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:55.767456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=876156edcf9643beaf74738d31177041)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:55.770035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:55.795918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:56.198081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '876156edcf9643beaf74738d31177041'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:56.272648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '876156edcf9643beaf74738d31177041'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:56.273411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '876156edcf9643beaf74738d31177041'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:56.281210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '876156edcf9643beaf74738d31177041'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:56.282022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '876156edcf9643beaf74738d31177041'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:56.299765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 921/1845] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:55 2022] DELETE /identity/v3/users/876156edcf9643beaf74738d31177041 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:56.321262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99476df1-4ae0-4717-9604-6c378fc62237 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:56.322305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99476df1-4ae0-4717-9604-6c378fc62237 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:56.323166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99476df1-4ae0-4717-9604-6c378fc62237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:56.352250 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-99476df1-4ae0-4717-9604-6c378fc62237 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:56.354493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-99476df1-4ae0-4717-9604-6c378fc62237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:56.455328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 925/1846] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:56.462671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:56.464393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:56.600952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07217ead-002d-4455-bef0-96cf9a3e0080 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:56.605245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60831498-b587-47bd-a21a-0903d48c55d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:56.606138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60831498-b587-47bd-a21a-0903d48c55d2 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:56.606425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60831498-b587-47bd-a21a-0903d48c55d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:56.606706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60831498-b587-47bd-a21a-0903d48c55d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:56.617772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60831498-b587-47bd-a21a-0903d48c55d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:56.619409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60831498-b587-47bd-a21a-0903d48c55d2 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:56.630200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 922/1847] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:56.702331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99476df1-4ae0-4717-9604-6c378fc62237 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:56.705030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-99476df1-4ae0-4717-9604-6c378fc62237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:56.813080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99476df1-4ae0-4717-9604-6c378fc62237 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:56.818370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0458f1-7c87-4449-b348-3961f965b081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:56.819381 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0458f1-7c87-4449-b348-3961f965b081 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:56.819875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0458f1-7c87-4449-b348-3961f965b081 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:56.820412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0458f1-7c87-4449-b348-3961f965b081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:56.822716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-de0458f1-7c87-4449-b348-3961f965b081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:56.894672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0458f1-7c87-4449-b348-3961f965b081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:56.898935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0458f1-7c87-4449-b348-3961f965b081 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:56.908225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 926/1848] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:55:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:55:57.310911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60831498-b587-47bd-a21a-0903d48c55d2 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:57.312417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-60831498-b587-47bd-a21a-0903d48c55d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:57.326624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60831498-b587-47bd-a21a-0903d48c55d2 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:57.331722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:57.332605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:57.332890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:57.333167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] PATH_INFO: `/v3/projects/ce8ab327172f416ea982936b287762d2` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:57.336027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce8ab327172f416ea982936b287762d2)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:57.338007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:57.512637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8ab327172f416ea982936b287762d2', 'request_id': 'req-b0e5918f-e137-4b34-aebd-17c4bd8796e1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:57.530310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8ab327172f416ea982936b287762d2', 'request_id': 'req-b0e5918f-e137-4b34-aebd-17c4bd8796e1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:57.609736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8ab327172f416ea982936b287762d2', 'request_id': 'req-b0e5918f-e137-4b34-aebd-17c4bd8796e1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:57.628252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] The token cache is being invalidate because project ce8ab327172f416ea982936b287762d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:57.629099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:57.630587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:57.636042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 923/1849] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:57 2022] DELETE /identity/v3/projects/ce8ab327172f416ea982936b287762d2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:57.639232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0458f1-7c87-4449-b348-3961f965b081 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:57.670683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0458f1-7c87-4449-b348-3961f965b081 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:57.695065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:57.757982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:57.757982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:57.757982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:57.757982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] PATH_INFO: `/v3/users/cfc4c09b0a8a4a5db9a4d330250fb2e2` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:57.758373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc4c09b0a8a4a5db9a4d330250fb2e2)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:57.760061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:58.057661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc4c09b0a8a4a5db9a4d330250fb2e2'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:58.205919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc4c09b0a8a4a5db9a4d330250fb2e2'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:58.206623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc4c09b0a8a4a5db9a4d330250fb2e2'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:58.222854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc4c09b0a8a4a5db9a4d330250fb2e2'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:58.222854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a75bea86-2505-4bfa-b17d-711675e18e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc4c09b0a8a4a5db9a4d330250fb2e2'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:58.236219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 927/1850] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:55:57 2022] DELETE /identity/v3/users/cfc4c09b0a8a4a5db9a4d330250fb2e2 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:59.034326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:59.043075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:59.073796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5918f-e137-4b34-aebd-17c4bd8796e1 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:59.089492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:59.146128 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:59.147635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:59.148022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:59.148406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] PATH_INFO: `/v3/projects/d57676cc6e9c453aad20204f6c6f33de` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:59.149837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57676cc6e9c453aad20204f6c6f33de)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:59.152715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:59.263803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57676cc6e9c453aad20204f6c6f33de', 'request_id': 'req-447317df-434a-4525-90dd-4e41b1f2a2d7'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:59.269356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57676cc6e9c453aad20204f6c6f33de', 'request_id': 'req-447317df-434a-4525-90dd-4e41b1f2a2d7'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:59.342725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57676cc6e9c453aad20204f6c6f33de', 'request_id': 'req-447317df-434a-4525-90dd-4e41b1f2a2d7'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:59.357513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] The token cache is being invalidate because project d57676cc6e9c453aad20204f6c6f33de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:55:59.357807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:59.358606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-447317df-434a-4525-90dd-4e41b1f2a2d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:55:59.360280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 924/1851] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:55:59 2022] DELETE /identity/v3/projects/d57676cc6e9c453aad20204f6c6f33de => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:55:59.365359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe468fd7-9b81-4812-aee2-9de4d7dd289b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:59.366350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe468fd7-9b81-4812-aee2-9de4d7dd289b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:59.366637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe468fd7-9b81-4812-aee2-9de4d7dd289b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:59.384982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-fe468fd7-9b81-4812-aee2-9de4d7dd289b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:59.386348 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fe468fd7-9b81-4812-aee2-9de4d7dd289b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:59.451215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 928/1852] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:55:59.459102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79fa61-1a1f-4d04-9b92-34e97230e26b None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:59.459415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79fa61-1a1f-4d04-9b92-34e97230e26b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:59.459704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79fa61-1a1f-4d04-9b92-34e97230e26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:59.480200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cf79fa61-1a1f-4d04-9b92-34e97230e26b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:55:59.481423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cf79fa61-1a1f-4d04-9b92-34e97230e26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:55:59.489556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 925/1853] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:55:59.494529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe468fd7-9b81-4812-aee2-9de4d7dd289b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:59.504359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe468fd7-9b81-4812-aee2-9de4d7dd289b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:59.510051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566dfd3-b1d9-42d8-891f-b67f8565963a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:59.513078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566dfd3-b1d9-42d8-891f-b67f8565963a admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:59.513078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566dfd3-b1d9-42d8-891f-b67f8565963a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:59.513078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566dfd3-b1d9-42d8-891f-b67f8565963a admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:59.513078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566dfd3-b1d9-42d8-891f-b67f8565963a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:55:59.513078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566dfd3-b1d9-42d8-891f-b67f8565963a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:59.513896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566dfd3-b1d9-42d8-891f-b67f8565963a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:59.526427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 929/1854] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:55:59.531358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79fa61-1a1f-4d04-9b92-34e97230e26b None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:59.541161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79fa61-1a1f-4d04-9b92-34e97230e26b None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:59.546138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e58630-9eb6-40ba-864f-a0c42a25948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:59.549308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e58630-9eb6-40ba-864f-a0c42a25948a admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:59.549308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e58630-9eb6-40ba-864f-a0c42a25948a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:59.549308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e58630-9eb6-40ba-864f-a0c42a25948a admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:59.549308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e58630-9eb6-40ba-864f-a0c42a25948a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:59.550057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e58630-9eb6-40ba-864f-a0c42a25948a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:59.642361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 926/1855] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:55:59 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:59.647015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566dfd3-b1d9-42d8-891f-b67f8565963a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:59.657790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566dfd3-b1d9-42d8-891f-b67f8565963a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:59.688102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:55:59.719932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30406e5b-e19b-44ba-abe4-b8a109de7721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:59.720628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30406e5b-e19b-44ba-abe4-b8a109de7721 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:59.720973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30406e5b-e19b-44ba-abe4-b8a109de7721 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:59.721237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30406e5b-e19b-44ba-abe4-b8a109de7721 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:59.721974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30406e5b-e19b-44ba-abe4-b8a109de7721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:59.723961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30406e5b-e19b-44ba-abe4-b8a109de7721 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:59.863156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 930/1856] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:55:59 2022] POST /identity/v3/users => generated 282 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:55:59.863452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e58630-9eb6-40ba-864f-a0c42a25948a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:59.874832 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e58630-9eb6-40ba-864f-a0c42a25948a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:59.879645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56258d19-cc1d-4ca4-95e7-4e4b701619f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:59.880357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56258d19-cc1d-4ca4-95e7-4e4b701619f2 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:59.884677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56258d19-cc1d-4ca4-95e7-4e4b701619f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:59.884677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56258d19-cc1d-4ca4-95e7-4e4b701619f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:59.884677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56258d19-cc1d-4ca4-95e7-4e4b701619f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:59.884677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56258d19-cc1d-4ca4-95e7-4e4b701619f2 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:59.894528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 927/1857] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:59.899619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30406e5b-e19b-44ba-abe4-b8a109de7721 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:59.918523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30406e5b-e19b-44ba-abe4-b8a109de7721 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:59.923630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69259c5d-2dcd-4a65-b300-1014cdd28991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:59.924352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69259c5d-2dcd-4a65-b300-1014cdd28991 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:59.924648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69259c5d-2dcd-4a65-b300-1014cdd28991 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:59.924953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69259c5d-2dcd-4a65-b300-1014cdd28991 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:59.925767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69259c5d-2dcd-4a65-b300-1014cdd28991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:59.928121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69259c5d-2dcd-4a65-b300-1014cdd28991 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:55:59.940889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 931/1858] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:55:59.945523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56258d19-cc1d-4ca4-95e7-4e4b701619f2 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:55:59.956863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56258d19-cc1d-4ca4-95e7-4e4b701619f2 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:55:59.962530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79312477-dee9-46fd-85eb-d5a1ce9c52dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:55:59.963519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79312477-dee9-46fd-85eb-d5a1ce9c52dd admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:55:59.963924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79312477-dee9-46fd-85eb-d5a1ce9c52dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:55:59.964234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79312477-dee9-46fd-85eb-d5a1ce9c52dd admin admin] PATH_INFO: `/v3/projects/84e3518005af4096b4ccf0789973c629/users/ab5ac0761aaf4a8da141a38b08a98700/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:55:59.980998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79312477-dee9-46fd-85eb-d5a1ce9c52dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e3518005af4096b4ccf0789973c629, user_id=ab5ac0761aaf4a8da141a38b08a98700, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:55:59.983793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79312477-dee9-46fd-85eb-d5a1ce9c52dd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:00.064375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 928/1859] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:55:59 2022] PUT /identity/v3/projects/84e3518005af4096b4ccf0789973c629/users/ab5ac0761aaf4a8da141a38b08a98700/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:00.073092 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e7f0b-1f13-41bb-9e2f-1887f86b84c2 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:00.073453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e7f0b-1f13-41bb-9e2f-1887f86b84c2 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:00.073875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e7f0b-1f13-41bb-9e2f-1887f86b84c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:00.103958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-828e7f0b-1f13-41bb-9e2f-1887f86b84c2 None None] MFA Rules not processed for user `ab5ac0761aaf4a8da141a38b08a98700`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:00.105173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-828e7f0b-1f13-41bb-9e2f-1887f86b84c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:00.181482 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 932/1860] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:00.189181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79312477-dee9-46fd-85eb-d5a1ce9c52dd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:00.246702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:00.268294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79312477-dee9-46fd-85eb-d5a1ce9c52dd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:00.273932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e986a33-a6cf-49eb-b2cd-b2fe3b17d709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:00.274618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e986a33-a6cf-49eb-b2cd-b2fe3b17d709 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:00.274905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e986a33-a6cf-49eb-b2cd-b2fe3b17d709 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:00.275216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e986a33-a6cf-49eb-b2cd-b2fe3b17d709 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:00.275920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e986a33-a6cf-49eb-b2cd-b2fe3b17d709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:00.277857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e986a33-a6cf-49eb-b2cd-b2fe3b17d709 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:00.426466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 929/1861] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:00 2022] POST /identity/v3/projects => generated 365 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:00.430613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e7f0b-1f13-41bb-9e2f-1887f86b84c2 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:00.441433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e7f0b-1f13-41bb-9e2f-1887f86b84c2 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:00.506661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0357eb4-c583-40e4-9862-ebd443d32322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:00.507364 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0357eb4-c583-40e4-9862-ebd443d32322 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:00.507689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0357eb4-c583-40e4-9862-ebd443d32322 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:00.509270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0357eb4-c583-40e4-9862-ebd443d32322 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:00.509270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0357eb4-c583-40e4-9862-ebd443d32322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:00.510751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0357eb4-c583-40e4-9862-ebd443d32322 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:00.617607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 933/1862] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:00 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:00.622028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e986a33-a6cf-49eb-b2cd-b2fe3b17d709 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:00.633075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e986a33-a6cf-49eb-b2cd-b2fe3b17d709 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:00.637672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b63388-57b4-4cdf-b776-bb5a7edc61eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:00.638386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b63388-57b4-4cdf-b776-bb5a7edc61eb admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:00.638678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b63388-57b4-4cdf-b776-bb5a7edc61eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:00.638956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b63388-57b4-4cdf-b776-bb5a7edc61eb admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:00.639681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b63388-57b4-4cdf-b776-bb5a7edc61eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:00.641323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b63388-57b4-4cdf-b776-bb5a7edc61eb admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:00.653489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 930/1863] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:00.657799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0357eb4-c583-40e4-9862-ebd443d32322 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:00.668642 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0357eb4-c583-40e4-9862-ebd443d32322 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:00.673328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450d140-a1b1-4a64-9633-a383a7ac843a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:00.674300 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450d140-a1b1-4a64-9633-a383a7ac843a admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:00.674582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450d140-a1b1-4a64-9633-a383a7ac843a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:00.674854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450d140-a1b1-4a64-9633-a383a7ac843a admin admin] PATH_INFO: `/v3/projects/bace5b9ab4c7485c891b23858ffe8ff2/users/451daa1e808343949bd936ddbedb7925/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:00.691065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450d140-a1b1-4a64-9633-a383a7ac843a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bace5b9ab4c7485c891b23858ffe8ff2, user_id=451daa1e808343949bd936ddbedb7925, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:00.694018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450d140-a1b1-4a64-9633-a383a7ac843a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:00.782825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 934/1864] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:00 2022] PUT /identity/v3/projects/bace5b9ab4c7485c891b23858ffe8ff2/users/451daa1e808343949bd936ddbedb7925/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:00.787244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b63388-57b4-4cdf-b776-bb5a7edc61eb admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:00.798366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b63388-57b4-4cdf-b776-bb5a7edc61eb admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:00.859395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de183e3-d4d3-4479-b75c-14966e79d2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:00.860090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de183e3-d4d3-4479-b75c-14966e79d2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:00.860371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de183e3-d4d3-4479-b75c-14966e79d2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:00.860646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de183e3-d4d3-4479-b75c-14966e79d2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:00.861390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de183e3-d4d3-4479-b75c-14966e79d2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:00.863019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de183e3-d4d3-4479-b75c-14966e79d2e5 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:00.873175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 931/1865] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:00.877672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450d140-a1b1-4a64-9633-a383a7ac843a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:00.883746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:00.889183 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450d140-a1b1-4a64-9633-a383a7ac843a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:00.899556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b919a7a-305c-4022-bef7-f6f4f0499135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:00.899556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b919a7a-305c-4022-bef7-f6f4f0499135 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:00.899556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b919a7a-305c-4022-bef7-f6f4f0499135 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:00.899556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b919a7a-305c-4022-bef7-f6f4f0499135 admin admin] PATH_INFO: `/v3/projects/bace5b9ab4c7485c891b23858ffe8ff2/users/451daa1e808343949bd936ddbedb7925/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:00.900062 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b919a7a-305c-4022-bef7-f6f4f0499135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bace5b9ab4c7485c891b23858ffe8ff2, user_id=451daa1e808343949bd936ddbedb7925, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:00.900383 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b919a7a-305c-4022-bef7-f6f4f0499135 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:00.910465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 935/1866] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:00 2022] PUT /identity/v3/projects/bace5b9ab4c7485c891b23858ffe8ff2/users/451daa1e808343949bd936ddbedb7925/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:00.921836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10619e0-fc06-42d9-be2e-90d2294057cc None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:00.922176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10619e0-fc06-42d9-be2e-90d2294057cc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:00.922478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10619e0-fc06-42d9-be2e-90d2294057cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:00.950455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d10619e0-fc06-42d9-be2e-90d2294057cc None None] MFA Rules not processed for user `451daa1e808343949bd936ddbedb7925`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:00.951683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d10619e0-fc06-42d9-be2e-90d2294057cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:01.019184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 932/1867] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:01.030526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b919a7a-305c-4022-bef7-f6f4f0499135 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:01.032263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0b919a7a-305c-4022-bef7-f6f4f0499135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:01.106298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b919a7a-305c-4022-bef7-f6f4f0499135 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:01.110153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1669b-5d04-4e12-b0a9-0482edce82fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:01.110819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1669b-5d04-4e12-b0a9-0482edce82fb service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:01.112349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1669b-5d04-4e12-b0a9-0482edce82fb service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:01.112641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1669b-5d04-4e12-b0a9-0482edce82fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:01.129338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1669b-5d04-4e12-b0a9-0482edce82fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:01.130825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1669b-5d04-4e12-b0a9-0482edce82fb service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:01.140489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 936/1868] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:01.363548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10619e0-fc06-42d9-be2e-90d2294057cc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:01.370320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d10619e0-fc06-42d9-be2e-90d2294057cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:01.386975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10619e0-fc06-42d9-be2e-90d2294057cc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:01.391380 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53425b-2d68-4fae-bf77-2fdb126805ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:01.392138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53425b-2d68-4fae-bf77-2fdb126805ab service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:01.392495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53425b-2d68-4fae-bf77-2fdb126805ab service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:01.392997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53425b-2d68-4fae-bf77-2fdb126805ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:01.436555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:01.465000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53425b-2d68-4fae-bf77-2fdb126805ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:01.466610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53425b-2d68-4fae-bf77-2fdb126805ab service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:01.477787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 933/1869] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:01 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:02.943057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1669b-5d04-4e12-b0a9-0482edce82fb service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:02.945105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-47d1669b-5d04-4e12-b0a9-0482edce82fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:03.014275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1669b-5d04-4e12-b0a9-0482edce82fb service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:03.018880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:03.019896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:03.020176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:03.020447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] PATH_INFO: `/v3/users/2d5c1ff0f9094c5a9cad57ca8413f38e` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:03.021687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5c1ff0f9094c5a9cad57ca8413f38e)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:03.023597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:03.308557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5c1ff0f9094c5a9cad57ca8413f38e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:03.400070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5c1ff0f9094c5a9cad57ca8413f38e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:03.401458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5c1ff0f9094c5a9cad57ca8413f38e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:03.413327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5c1ff0f9094c5a9cad57ca8413f38e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:03.414397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5c1ff0f9094c5a9cad57ca8413f38e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:03.443315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 937/1870] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:02 2022] DELETE /identity/v3/users/2d5c1ff0f9094c5a9cad57ca8413f38e => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:03.448558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccb4ea-4eba-487a-9db2-89e6ef14620a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:03.449007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccb4ea-4eba-487a-9db2-89e6ef14620a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:03.449377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daccb4ea-4eba-487a-9db2-89e6ef14620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:03.467394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-daccb4ea-4eba-487a-9db2-89e6ef14620a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:03.468632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-daccb4ea-4eba-487a-9db2-89e6ef14620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:03.528982 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 934/1871] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:03.539310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:03.544180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:03.554611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:03.597537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b8422-8a1b-43c9-a574-9afb761388b0 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:03.601688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f87be4-9781-4c73-bcd7-be1ffa0ca7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:03.602373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f87be4-9781-4c73-bcd7-be1ffa0ca7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:03.602659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f87be4-9781-4c73-bcd7-be1ffa0ca7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:03.602940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f87be4-9781-4c73-bcd7-be1ffa0ca7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:03.611965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f87be4-9781-4c73-bcd7-be1ffa0ca7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:03.613331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f87be4-9781-4c73-bcd7-be1ffa0ca7f5 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:03.624208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 938/1872] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:04.143311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccb4ea-4eba-487a-9db2-89e6ef14620a None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:04.144752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-daccb4ea-4eba-487a-9db2-89e6ef14620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:04.157418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daccb4ea-4eba-487a-9db2-89e6ef14620a None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:04.162899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:04.164125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:04.164545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:04.164898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] PATH_INFO: `/v3/projects/f0b84ea38ce4408aa417b5593728e745` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:04.166391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b84ea38ce4408aa417b5593728e745)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:04.168336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:04.317714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b84ea38ce4408aa417b5593728e745', 'request_id': 'req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.323940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b84ea38ce4408aa417b5593728e745', 'request_id': 'req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.411215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b84ea38ce4408aa417b5593728e745', 'request_id': 'req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.423233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] The token cache is being invalidate because project f0b84ea38ce4408aa417b5593728e745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:04.423488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.424127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.425722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 935/1873] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:04 2022] DELETE /identity/v3/projects/f0b84ea38ce4408aa417b5593728e745 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:04.438847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f87be4-9781-4c73-bcd7-be1ffa0ca7f5 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:04.449361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f87be4-9781-4c73-bcd7-be1ffa0ca7f5 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:04.514433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:04.515322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:04.515607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:04.515884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] PATH_INFO: `/v3/users/95debba9747f498a90bed24a0273d773` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:04.517277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95debba9747f498a90bed24a0273d773)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:04.519463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:04.667327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95debba9747f498a90bed24a0273d773'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.732002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95debba9747f498a90bed24a0273d773'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.732920 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95debba9747f498a90bed24a0273d773'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.741884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95debba9747f498a90bed24a0273d773'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.742742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95debba9747f498a90bed24a0273d773'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:04.761664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 939/1874] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:04 2022] DELETE /identity/v3/users/95debba9747f498a90bed24a0273d773 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:05.550087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:05.551934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:05.572812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa8b63-9c99-4bfb-ac69-75daccc001b0 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:05.632244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:05.633110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:05.633417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:05.633693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] PATH_INFO: `/v3/projects/8a913be2eaaa4a94b84c315b8d61db02` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:05.634908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a913be2eaaa4a94b84c315b8d61db02)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:05.636854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:05.638348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:05.732497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a913be2eaaa4a94b84c315b8d61db02', 'request_id': 'req-32435763-11c2-4a6d-914b-be762e4c7c20'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:05.738936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a913be2eaaa4a94b84c315b8d61db02', 'request_id': 'req-32435763-11c2-4a6d-914b-be762e4c7c20'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:05.863169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a913be2eaaa4a94b84c315b8d61db02', 'request_id': 'req-32435763-11c2-4a6d-914b-be762e4c7c20'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:05.876885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] The token cache is being invalidate because project 8a913be2eaaa4a94b84c315b8d61db02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:05.877216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:05.878038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:05.880908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 936/1875] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:05 2022] DELETE /identity/v3/projects/8a913be2eaaa4a94b84c315b8d61db02 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:05.884784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:05.886346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:05.901516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a6a0d-68e5-4002-9ba1-459994e937f8 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:05.937152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:05.965272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:05.966114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:05.966434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:05.966710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] PATH_INFO: `/v3/users/1efab6f85f2847c7a8ee4dff26b5c009` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:05.967885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1efab6f85f2847c7a8ee4dff26b5c009)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:05.969798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:06.169802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1efab6f85f2847c7a8ee4dff26b5c009'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:06.260755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1efab6f85f2847c7a8ee4dff26b5c009'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:06.261570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1efab6f85f2847c7a8ee4dff26b5c009'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:06.270225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1efab6f85f2847c7a8ee4dff26b5c009'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:06.271054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87548811-42ef-42fa-b241-64ff19bb7685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1efab6f85f2847c7a8ee4dff26b5c009'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:06.311804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 940/1876] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:05 2022] DELETE /identity/v3/users/1efab6f85f2847c7a8ee4dff26b5c009 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:06.857346 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:06.858945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:06.875031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32435763-11c2-4a6d-914b-be762e4c7c20 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:06.949428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:06.950363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:06.950663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:06.950990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] PATH_INFO: `/v3/projects/fa2bedfd90f8423785a23358e3a404b5` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:06.952281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2bedfd90f8423785a23358e3a404b5)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:06.954360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:07.105240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2bedfd90f8423785a23358e3a404b5', 'request_id': 'req-74b3cb3e-f637-4356-a974-51c4f1f230d2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:07.110528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2bedfd90f8423785a23358e3a404b5', 'request_id': 'req-74b3cb3e-f637-4356-a974-51c4f1f230d2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:07.188192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2bedfd90f8423785a23358e3a404b5', 'request_id': 'req-74b3cb3e-f637-4356-a974-51c4f1f230d2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:07.200424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] The token cache is being invalidate because project fa2bedfd90f8423785a23358e3a404b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:07.200715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:07.201440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-74b3cb3e-f637-4356-a974-51c4f1f230d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:07.203846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 937/1877] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:06 2022] DELETE /identity/v3/projects/fa2bedfd90f8423785a23358e3a404b5 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:07.210446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd30775-4adb-48bf-9d3a-5e1d770fb14e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:07.211252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd30775-4adb-48bf-9d3a-5e1d770fb14e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:07.211540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd30775-4adb-48bf-9d3a-5e1d770fb14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:07.230372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ddd30775-4adb-48bf-9d3a-5e1d770fb14e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:07.231752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ddd30775-4adb-48bf-9d3a-5e1d770fb14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:07.310484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 941/1878] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:07.312327 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caae159-6482-44de-a635-2d10b7654f5e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:07.312746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caae159-6482-44de-a635-2d10b7654f5e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:07.313092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caae159-6482-44de-a635-2d10b7654f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:07.332376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-5caae159-6482-44de-a635-2d10b7654f5e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:07.333713 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5caae159-6482-44de-a635-2d10b7654f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:07.342427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 938/1879] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:07.346746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd30775-4adb-48bf-9d3a-5e1d770fb14e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:07.358587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd30775-4adb-48bf-9d3a-5e1d770fb14e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:07.364049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f926e-8cfe-4dff-95b8-aa420586ef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:07.364901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f926e-8cfe-4dff-95b8-aa420586ef49 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:07.365326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f926e-8cfe-4dff-95b8-aa420586ef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:07.365706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f926e-8cfe-4dff-95b8-aa420586ef49 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:07.366594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f926e-8cfe-4dff-95b8-aa420586ef49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:56:07.367007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f926e-8cfe-4dff-95b8-aa420586ef49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:07.368744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f926e-8cfe-4dff-95b8-aa420586ef49 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:07.386166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 942/1880] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:56:07.387669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caae159-6482-44de-a635-2d10b7654f5e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:07.399558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caae159-6482-44de-a635-2d10b7654f5e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:07.404569 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6f174-1be2-4f7e-9f5e-e9e5ccb163bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:07.405269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6f174-1be2-4f7e-9f5e-e9e5ccb163bb admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:07.405551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6f174-1be2-4f7e-9f5e-e9e5ccb163bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:07.405951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6f174-1be2-4f7e-9f5e-e9e5ccb163bb admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:07.406580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6f174-1be2-4f7e-9f5e-e9e5ccb163bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:07.408754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6f174-1be2-4f7e-9f5e-e9e5ccb163bb admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:07.420875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:07.529165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 939/1881] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:07 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:07.530401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f926e-8cfe-4dff-95b8-aa420586ef49 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:07.540657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f926e-8cfe-4dff-95b8-aa420586ef49 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:07.581712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:07.601227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85629f26-851b-4974-b515-4e8caad508dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:07.601953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85629f26-851b-4974-b515-4e8caad508dd admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:07.602243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85629f26-851b-4974-b515-4e8caad508dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:07.602523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85629f26-851b-4974-b515-4e8caad508dd admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:07.603219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85629f26-851b-4974-b515-4e8caad508dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:07.605145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85629f26-851b-4974-b515-4e8caad508dd admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:07.700953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 943/1882] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:07 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:07.702314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6f174-1be2-4f7e-9f5e-e9e5ccb163bb admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:07.712625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6f174-1be2-4f7e-9f5e-e9e5ccb163bb admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:07.717344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82a866-f9ce-4fa8-8c94-bddbd95f5f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:07.718180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82a866-f9ce-4fa8-8c94-bddbd95f5f96 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:07.718658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82a866-f9ce-4fa8-8c94-bddbd95f5f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:07.718938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82a866-f9ce-4fa8-8c94-bddbd95f5f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:07.720143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82a866-f9ce-4fa8-8c94-bddbd95f5f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:07.722404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82a866-f9ce-4fa8-8c94-bddbd95f5f96 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:07.735228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 940/1883] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:07.739439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85629f26-851b-4974-b515-4e8caad508dd admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:07.751576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85629f26-851b-4974-b515-4e8caad508dd admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:07.757276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7535186-cee0-4112-8b51-bf6bc5af79bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:07.758011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7535186-cee0-4112-8b51-bf6bc5af79bf admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:07.760211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7535186-cee0-4112-8b51-bf6bc5af79bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:07.760211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7535186-cee0-4112-8b51-bf6bc5af79bf admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:07.760211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7535186-cee0-4112-8b51-bf6bc5af79bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:07.760942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7535186-cee0-4112-8b51-bf6bc5af79bf admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:07.771967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 944/1884] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:56:07.775922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82a866-f9ce-4fa8-8c94-bddbd95f5f96 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:07.786377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82a866-f9ce-4fa8-8c94-bddbd95f5f96 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:07.792150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4802994-94a2-4d6e-8776-7367acee9995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:07.793054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4802994-94a2-4d6e-8776-7367acee9995 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:07.793346 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4802994-94a2-4d6e-8776-7367acee9995 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:07.793637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4802994-94a2-4d6e-8776-7367acee9995 admin admin] PATH_INFO: `/v3/projects/6494fe023ccc42ea99548ad614cf00b4/users/7f5ec3a96aaa43c18f48b4ac732ee86a/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:07.810845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4802994-94a2-4d6e-8776-7367acee9995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6494fe023ccc42ea99548ad614cf00b4, user_id=7f5ec3a96aaa43c18f48b4ac732ee86a, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:07.813992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4802994-94a2-4d6e-8776-7367acee9995 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:07.884146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 941/1885] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:07 2022] PUT /identity/v3/projects/6494fe023ccc42ea99548ad614cf00b4/users/7f5ec3a96aaa43c18f48b4ac732ee86a/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:07.895520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214aefe7-f41a-47c5-a275-217d44c7f9f5 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:07.895818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214aefe7-f41a-47c5-a275-217d44c7f9f5 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:07.896101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214aefe7-f41a-47c5-a275-217d44c7f9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:07.924122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-214aefe7-f41a-47c5-a275-217d44c7f9f5 None None] MFA Rules not processed for user `7f5ec3a96aaa43c18f48b4ac732ee86a`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:07.925462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-214aefe7-f41a-47c5-a275-217d44c7f9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:07.993079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 945/1886] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:08.005409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4802994-94a2-4d6e-8776-7367acee9995 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:08.007264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b4802994-94a2-4d6e-8776-7367acee9995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:08.085289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4802994-94a2-4d6e-8776-7367acee9995 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:08.090609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e0617-595e-4413-a635-fe7a49b9352d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:08.091388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e0617-595e-4413-a635-fe7a49b9352d service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:08.091759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e0617-595e-4413-a635-fe7a49b9352d service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:08.092135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e0617-595e-4413-a635-fe7a49b9352d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:08.104233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e0617-595e-4413-a635-fe7a49b9352d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:08.106069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e0617-595e-4413-a635-fe7a49b9352d service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:08.130055 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 942/1887] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:08.147588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214aefe7-f41a-47c5-a275-217d44c7f9f5 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:08.212846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214aefe7-f41a-47c5-a275-217d44c7f9f5 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:08.217471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:08.218499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:08.218786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:08.219066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] PATH_INFO: `/v3/users/7f5ec3a96aaa43c18f48b4ac732ee86a` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:08.220279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5ec3a96aaa43c18f48b4ac732ee86a)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:08.222247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:08.448069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5ec3a96aaa43c18f48b4ac732ee86a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:08.451091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:08.581875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5ec3a96aaa43c18f48b4ac732ee86a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:08.582524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5ec3a96aaa43c18f48b4ac732ee86a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:08.590413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5ec3a96aaa43c18f48b4ac732ee86a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:08.591129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5ec3a96aaa43c18f48b4ac732ee86a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:08.622559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 946/1888] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:08 2022] DELETE /identity/v3/users/7f5ec3a96aaa43c18f48b4ac732ee86a => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:08.628898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0b07c-589b-4a08-96e7-e940b0eed162 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:08.629427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0b07c-589b-4a08-96e7-e940b0eed162 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:08.629726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0b07c-589b-4a08-96e7-e940b0eed162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:08.650145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-35a0b07c-589b-4a08-96e7-e940b0eed162 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:08.652070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-35a0b07c-589b-4a08-96e7-e940b0eed162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:08.726810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 943/1889] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:08.738427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:08.739836 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:08.801374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2a6c3-c1d5-4568-8f71-d34b9e569b3b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:08.805580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd45dd2-64e4-4ef0-8161-89293d3fedb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:08.806277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd45dd2-64e4-4ef0-8161-89293d3fedb0 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:08.806569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd45dd2-64e4-4ef0-8161-89293d3fedb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:08.806880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd45dd2-64e4-4ef0-8161-89293d3fedb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:08.816591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd45dd2-64e4-4ef0-8161-89293d3fedb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:08.818073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd45dd2-64e4-4ef0-8161-89293d3fedb0 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:08.828044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 947/1890] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:09.282423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0b07c-589b-4a08-96e7-e940b0eed162 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:09.284176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-35a0b07c-589b-4a08-96e7-e940b0eed162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:09.302673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0b07c-589b-4a08-96e7-e940b0eed162 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:09.309072 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:09.311788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:09.312502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:09.312657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] PATH_INFO: `/v3/projects/6494fe023ccc42ea99548ad614cf00b4` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:09.314213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6494fe023ccc42ea99548ad614cf00b4)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:09.316087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:09.443773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6494fe023ccc42ea99548ad614cf00b4', 'request_id': 'req-c33aeab2-b3a1-482a-be4d-8b928aa021b2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:09.447747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:09.450011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6494fe023ccc42ea99548ad614cf00b4', 'request_id': 'req-c33aeab2-b3a1-482a-be4d-8b928aa021b2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:09.523779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6494fe023ccc42ea99548ad614cf00b4', 'request_id': 'req-c33aeab2-b3a1-482a-be4d-8b928aa021b2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:09.535028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] The token cache is being invalidate because project 6494fe023ccc42ea99548ad614cf00b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:09.535321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:09.536242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c33aeab2-b3a1-482a-be4d-8b928aa021b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:09.538956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 944/1891] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:09 2022] DELETE /identity/v3/projects/6494fe023ccc42ea99548ad614cf00b4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:14.729218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd45dd2-64e4-4ef0-8161-89293d3fedb0 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:14.730677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-afd45dd2-64e4-4ef0-8161-89293d3fedb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:14.744193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd45dd2-64e4-4ef0-8161-89293d3fedb0 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:14.800648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:14.801711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:14.802025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:14.802304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] PATH_INFO: `/v3/users/ab5ac0761aaf4a8da141a38b08a98700` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:14.804001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5ac0761aaf4a8da141a38b08a98700)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:14.805960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:15.031084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5ac0761aaf4a8da141a38b08a98700'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:15.102828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5ac0761aaf4a8da141a38b08a98700'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:15.103408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5ac0761aaf4a8da141a38b08a98700'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:15.111105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5ac0761aaf4a8da141a38b08a98700'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:15.111738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5ac0761aaf4a8da141a38b08a98700'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:15.119409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:15.130394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 948/1892] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:14 2022] DELETE /identity/v3/users/ab5ac0761aaf4a8da141a38b08a98700 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:15.135646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7959c7-e530-40b8-a255-65db0ed949c8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:15.136109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7959c7-e530-40b8-a255-65db0ed949c8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:15.136229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7959c7-e530-40b8-a255-65db0ed949c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:15.153508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ea7959c7-e530-40b8-a255-65db0ed949c8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:15.154732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ea7959c7-e530-40b8-a255-65db0ed949c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:15.217694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 945/1893] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:15.228389 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:15.229459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:15.285665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c910c-970f-4a36-be90-ab2c4f0385e6 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:15.289755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056a40d-c5ed-4255-a246-7f51d52a6a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:15.290432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a40d-c5ed-4255-a246-7f51d52a6a67 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:15.290717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a40d-c5ed-4255-a246-7f51d52a6a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:15.290995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056a40d-c5ed-4255-a246-7f51d52a6a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:15.300088 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056a40d-c5ed-4255-a246-7f51d52a6a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:15.301798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056a40d-c5ed-4255-a246-7f51d52a6a67 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:15.312045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 949/1894] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:15.698498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7959c7-e530-40b8-a255-65db0ed949c8 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:15.699940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ea7959c7-e530-40b8-a255-65db0ed949c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:15.712700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7959c7-e530-40b8-a255-65db0ed949c8 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:15.718090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:15.720291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:15.720579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:15.720862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] PATH_INFO: `/v3/projects/84e3518005af4096b4ccf0789973c629` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:15.722137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e3518005af4096b4ccf0789973c629)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:15.731798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:15.836136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e3518005af4096b4ccf0789973c629', 'request_id': 'req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:15.842323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e3518005af4096b4ccf0789973c629', 'request_id': 'req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:15.999588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e3518005af4096b4ccf0789973c629', 'request_id': 'req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:16.011892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] The token cache is being invalidate because project 84e3518005af4096b4ccf0789973c629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:16.012182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:16.012809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:16.014393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 946/1895] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:15 2022] DELETE /identity/v3/projects/84e3518005af4096b4ccf0789973c629 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:16.018658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056a40d-c5ed-4255-a246-7f51d52a6a67 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:16.028850 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056a40d-c5ed-4255-a246-7f51d52a6a67 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:16.087946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:16.088821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:16.089104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:16.089382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] PATH_INFO: `/v3/users/451daa1e808343949bd936ddbedb7925` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:16.090595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=451daa1e808343949bd936ddbedb7925)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:16.092602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:16.299485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451daa1e808343949bd936ddbedb7925'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:16.376651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451daa1e808343949bd936ddbedb7925'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:16.377281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451daa1e808343949bd936ddbedb7925'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:16.385694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451daa1e808343949bd936ddbedb7925'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:16.386467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b08bc914-8019-4067-9064-f47a0275fa1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451daa1e808343949bd936ddbedb7925'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:16.404754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 950/1896] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:16 2022] DELETE /identity/v3/users/451daa1e808343949bd936ddbedb7925 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:16.875554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:16.877067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:16.891526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df95e5-b1e3-42bf-8d71-5d91c89de3f9 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:16.967305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:16.968190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:16.968475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:16.968758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] PATH_INFO: `/v3/projects/bace5b9ab4c7485c891b23858ffe8ff2` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:16.970030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bace5b9ab4c7485c891b23858ffe8ff2)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:16.971937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:16.980911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:17.084170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bace5b9ab4c7485c891b23858ffe8ff2', 'request_id': 'req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:17.089687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bace5b9ab4c7485c891b23858ffe8ff2', 'request_id': 'req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:17.177384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bace5b9ab4c7485c891b23858ffe8ff2', 'request_id': 'req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:17.190023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] The token cache is being invalidate because project bace5b9ab4c7485c891b23858ffe8ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:17.190305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:17.190954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-91dfd57a-58ac-4f25-90c1-6945f43c8ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:17.192559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 947/1897] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:16 2022] DELETE /identity/v3/projects/bace5b9ab4c7485c891b23858ffe8ff2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:17.198042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a506-f15f-4355-8561-d98a1a243dbd None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:17.198491 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a506-f15f-4355-8561-d98a1a243dbd None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:17.198634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a506-f15f-4355-8561-d98a1a243dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:17.216064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d021a506-f15f-4355-8561-d98a1a243dbd None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:17.217260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d021a506-f15f-4355-8561-d98a1a243dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:17.222535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:17.288710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 951/1898] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:17.296807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e533c4e-4453-434c-bd0e-064c13d8697c None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:17.297105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e533c4e-4453-434c-bd0e-064c13d8697c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:17.297383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e533c4e-4453-434c-bd0e-064c13d8697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:17.316249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8e533c4e-4453-434c-bd0e-064c13d8697c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:17.317441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8e533c4e-4453-434c-bd0e-064c13d8697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:17.325763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 948/1899] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:17.330075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021a506-f15f-4355-8561-d98a1a243dbd None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:17.339707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021a506-f15f-4355-8561-d98a1a243dbd None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:17.344267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e1039-25af-4399-9b02-1d733224c638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:17.344929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e1039-25af-4399-9b02-1d733224c638 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:17.345216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e1039-25af-4399-9b02-1d733224c638 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:17.345495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e1039-25af-4399-9b02-1d733224c638 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:17.346257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e1039-25af-4399-9b02-1d733224c638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:56:17.346598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e1039-25af-4399-9b02-1d733224c638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:17.348488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e1039-25af-4399-9b02-1d733224c638 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:17.360495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 952/1900] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:56:17.387374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e533c4e-4453-434c-bd0e-064c13d8697c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:17.396517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e533c4e-4453-434c-bd0e-064c13d8697c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:17.401120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d307d-544a-4c74-b771-8e0e51a7e208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:17.401777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d307d-544a-4c74-b771-8e0e51a7e208 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:17.402093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d307d-544a-4c74-b771-8e0e51a7e208 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:17.402380 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d307d-544a-4c74-b771-8e0e51a7e208 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:17.403141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d307d-544a-4c74-b771-8e0e51a7e208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:17.405051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d307d-544a-4c74-b771-8e0e51a7e208 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:17.521578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 949/1901] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:17 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:17.525840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e1039-25af-4399-9b02-1d733224c638 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:17.535808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e1039-25af-4399-9b02-1d733224c638 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:17.593203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17dd229-ee29-4a44-b56e-759c336317f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:17.593940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dd229-ee29-4a44-b56e-759c336317f3 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:17.594230 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dd229-ee29-4a44-b56e-759c336317f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:17.594513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17dd229-ee29-4a44-b56e-759c336317f3 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:17.595339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17dd229-ee29-4a44-b56e-759c336317f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:17.597266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17dd229-ee29-4a44-b56e-759c336317f3 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:17.680409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 953/1902] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:17 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:17.684946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d307d-544a-4c74-b771-8e0e51a7e208 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:17.695195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d307d-544a-4c74-b771-8e0e51a7e208 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:17.699797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910cae5-e5c1-401e-8b62-c4a0c2db3ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:17.700506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910cae5-e5c1-401e-8b62-c4a0c2db3ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:17.700783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910cae5-e5c1-401e-8b62-c4a0c2db3ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:17.701057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910cae5-e5c1-401e-8b62-c4a0c2db3ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:17.701784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910cae5-e5c1-401e-8b62-c4a0c2db3ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:17.703411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910cae5-e5c1-401e-8b62-c4a0c2db3ea3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:17.714257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 950/1903] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:17.718598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17dd229-ee29-4a44-b56e-759c336317f3 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:17.730190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17dd229-ee29-4a44-b56e-759c336317f3 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:17.738024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693074e-ebde-431e-ab88-795bbb2253a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:17.738425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693074e-ebde-431e-ab88-795bbb2253a4 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:17.738804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693074e-ebde-431e-ab88-795bbb2253a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:17.739176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693074e-ebde-431e-ab88-795bbb2253a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:17.740152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693074e-ebde-431e-ab88-795bbb2253a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:17.742619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693074e-ebde-431e-ab88-795bbb2253a4 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:17.756665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 954/1904] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:17.758103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910cae5-e5c1-401e-8b62-c4a0c2db3ea3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:17.768058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910cae5-e5c1-401e-8b62-c4a0c2db3ea3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:17.773263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07eec1-cac4-46e8-acf6-720d332e3e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:17.774178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07eec1-cac4-46e8-acf6-720d332e3e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:17.774539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07eec1-cac4-46e8-acf6-720d332e3e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:17.774815 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07eec1-cac4-46e8-acf6-720d332e3e1c admin admin] PATH_INFO: `/v3/projects/6b1291d3b2234563bfb45357253e97a7/users/f68c44881b194ffdaf57040c28e124d4/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:17.791113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07eec1-cac4-46e8-acf6-720d332e3e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b1291d3b2234563bfb45357253e97a7, user_id=f68c44881b194ffdaf57040c28e124d4, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:17.793747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd07eec1-cac4-46e8-acf6-720d332e3e1c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:17.898069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 951/1905] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:17 2022] PUT /identity/v3/projects/6b1291d3b2234563bfb45357253e97a7/users/f68c44881b194ffdaf57040c28e124d4/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:17.908476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bac34-6063-41d2-b2c8-08a4d44ecd29 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:17.908769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bac34-6063-41d2-b2c8-08a4d44ecd29 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:17.909050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bac34-6063-41d2-b2c8-08a4d44ecd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:17.941379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-3d1bac34-6063-41d2-b2c8-08a4d44ecd29 None None] MFA Rules not processed for user `f68c44881b194ffdaf57040c28e124d4`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:17.942702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3d1bac34-6063-41d2-b2c8-08a4d44ecd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:18.017866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 955/1906] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:18.023343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07eec1-cac4-46e8-acf6-720d332e3e1c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:18.105639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07eec1-cac4-46e8-acf6-720d332e3e1c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:18.111353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330e75-111a-4b9e-97d6-5c4471dc4385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:18.112168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330e75-111a-4b9e-97d6-5c4471dc4385 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:18.112454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330e75-111a-4b9e-97d6-5c4471dc4385 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:18.112739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330e75-111a-4b9e-97d6-5c4471dc4385 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:18.113438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61330e75-111a-4b9e-97d6-5c4471dc4385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:18.115422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61330e75-111a-4b9e-97d6-5c4471dc4385 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:18.205508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 952/1907] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:18 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:18.209477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bac34-6063-41d2-b2c8-08a4d44ecd29 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:18.221386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bac34-6063-41d2-b2c8-08a4d44ecd29 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:18.227181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:18.283722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2839ab-2f16-4f1c-8210-78c497fe5f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:18.284412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2839ab-2f16-4f1c-8210-78c497fe5f7e admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:18.284714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2839ab-2f16-4f1c-8210-78c497fe5f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:18.284969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2839ab-2f16-4f1c-8210-78c497fe5f7e admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:18.285660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2839ab-2f16-4f1c-8210-78c497fe5f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:18.287750 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2839ab-2f16-4f1c-8210-78c497fe5f7e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:18.409567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 956/1908] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:18 2022] POST /identity/v3/users => generated 298 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:18.414361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330e75-111a-4b9e-97d6-5c4471dc4385 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:18.425078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330e75-111a-4b9e-97d6-5c4471dc4385 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:18.431727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444d99d-aec0-4ab2-b049-84eac67fb390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:18.432443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444d99d-aec0-4ab2-b049-84eac67fb390 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:18.434904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444d99d-aec0-4ab2-b049-84eac67fb390 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:18.434904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444d99d-aec0-4ab2-b049-84eac67fb390 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:18.434904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444d99d-aec0-4ab2-b049-84eac67fb390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:18.435393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444d99d-aec0-4ab2-b049-84eac67fb390 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:18.452160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 953/1909] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:56:18.453139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2839ab-2f16-4f1c-8210-78c497fe5f7e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:18.465198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2839ab-2f16-4f1c-8210-78c497fe5f7e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:18.470299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0f99b-ece0-490a-982f-5e64f831052c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:18.471307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0f99b-ece0-490a-982f-5e64f831052c admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:18.471663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0f99b-ece0-490a-982f-5e64f831052c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:18.472019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0f99b-ece0-490a-982f-5e64f831052c admin admin] PATH_INFO: `/v3/projects/7e3f9f8abdbe46988d725678b154bea1/users/a9a7be01a1704140a9efe6f067048074/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:18.493012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0f99b-ece0-490a-982f-5e64f831052c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3f9f8abdbe46988d725678b154bea1, user_id=a9a7be01a1704140a9efe6f067048074, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:18.498181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0f99b-ece0-490a-982f-5e64f831052c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:18.594513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 957/1910] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:18 2022] PUT /identity/v3/projects/7e3f9f8abdbe46988d725678b154bea1/users/a9a7be01a1704140a9efe6f067048074/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:18.598450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444d99d-aec0-4ab2-b049-84eac67fb390 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:18.609010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444d99d-aec0-4ab2-b049-84eac67fb390 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:18.612576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:18.668753 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7830b86-8803-449d-aa7d-d46af3e7d75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:18.669467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830b86-8803-449d-aa7d-d46af3e7d75e admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:18.669751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830b86-8803-449d-aa7d-d46af3e7d75e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:18.670060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7830b86-8803-449d-aa7d-d46af3e7d75e admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:18.670791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7830b86-8803-449d-aa7d-d46af3e7d75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:18.672437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7830b86-8803-449d-aa7d-d46af3e7d75e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:18.682600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 954/1911] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:18.686924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0f99b-ece0-490a-982f-5e64f831052c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:18.697103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0f99b-ece0-490a-982f-5e64f831052c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:18.701658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477029c-7208-4d5e-a154-4c2fea89b746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:18.702567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477029c-7208-4d5e-a154-4c2fea89b746 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:18.702860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477029c-7208-4d5e-a154-4c2fea89b746 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:18.703134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477029c-7208-4d5e-a154-4c2fea89b746 admin admin] PATH_INFO: `/v3/projects/7e3f9f8abdbe46988d725678b154bea1/users/a9a7be01a1704140a9efe6f067048074/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:18.705175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477029c-7208-4d5e-a154-4c2fea89b746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3f9f8abdbe46988d725678b154bea1, user_id=a9a7be01a1704140a9efe6f067048074, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:18.707792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477029c-7208-4d5e-a154-4c2fea89b746 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:18.718595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 958/1912] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:18 2022] PUT /identity/v3/projects/7e3f9f8abdbe46988d725678b154bea1/users/a9a7be01a1704140a9efe6f067048074/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:18.730428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d49dc8-064d-4a4d-a78c-62a3b66d4be5 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:18.730730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d49dc8-064d-4a4d-a78c-62a3b66d4be5 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:18.731006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d49dc8-064d-4a4d-a78c-62a3b66d4be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:18.756934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-46d49dc8-064d-4a4d-a78c-62a3b66d4be5 None None] MFA Rules not processed for user `a9a7be01a1704140a9efe6f067048074`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:18.758192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-46d49dc8-064d-4a4d-a78c-62a3b66d4be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:18.829091 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 955/1913] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:18.840266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477029c-7208-4d5e-a154-4c2fea89b746 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:18.841957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6477029c-7208-4d5e-a154-4c2fea89b746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:18.901491 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477029c-7208-4d5e-a154-4c2fea89b746 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:18.905664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701eea5-8fb7-41f1-9418-09ff5057bb3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:18.906361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701eea5-8fb7-41f1-9418-09ff5057bb3f service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:18.906643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701eea5-8fb7-41f1-9418-09ff5057bb3f service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:18.906920 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701eea5-8fb7-41f1-9418-09ff5057bb3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:18.917135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701eea5-8fb7-41f1-9418-09ff5057bb3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:18.918662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701eea5-8fb7-41f1-9418-09ff5057bb3f service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:18.928835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 959/1914] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:19.026819 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d49dc8-064d-4a4d-a78c-62a3b66d4be5 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:19.028388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-46d49dc8-064d-4a4d-a78c-62a3b66d4be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:19.041660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d49dc8-064d-4a4d-a78c-62a3b66d4be5 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:19.045209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33d140-a0db-4466-bd92-fae7fd916584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:19.045898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33d140-a0db-4466-bd92-fae7fd916584 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:19.046184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33d140-a0db-4466-bd92-fae7fd916584 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:19.046460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33d140-a0db-4466-bd92-fae7fd916584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:19.108066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33d140-a0db-4466-bd92-fae7fd916584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:19.109498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33d140-a0db-4466-bd92-fae7fd916584 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:19.119212 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 956/1915] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:22.819338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701eea5-8fb7-41f1-9418-09ff5057bb3f service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:22.891265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701eea5-8fb7-41f1-9418-09ff5057bb3f service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:22.896308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:22.897300 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:22.897649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:22.898025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] PATH_INFO: `/v3/users/f68c44881b194ffdaf57040c28e124d4` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:22.899473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f68c44881b194ffdaf57040c28e124d4)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:22.901457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:22.904653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:23.117219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f68c44881b194ffdaf57040c28e124d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:23.194244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f68c44881b194ffdaf57040c28e124d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:23.194887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f68c44881b194ffdaf57040c28e124d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:23.203271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f68c44881b194ffdaf57040c28e124d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:23.204014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f68c44881b194ffdaf57040c28e124d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:23.223463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 960/1916] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:22 2022] DELETE /identity/v3/users/f68c44881b194ffdaf57040c28e124d4 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:23.228887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c47005-a4dc-4ad7-80e0-ad7572f930e8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:23.232097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c47005-a4dc-4ad7-80e0-ad7572f930e8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:23.232097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c47005-a4dc-4ad7-80e0-ad7572f930e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:23.249155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b5c47005-a4dc-4ad7-80e0-ad7572f930e8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:23.250433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b5c47005-a4dc-4ad7-80e0-ad7572f930e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:23.273142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:23.313549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 957/1917] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:23.323681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:23.325213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:23.388146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b10dd-8159-4c02-93c2-d2042aecad42 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:23.391727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b57ad-5606-42e8-bffc-7cf349f5a3d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:23.392390 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b57ad-5606-42e8-bffc-7cf349f5a3d7 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:23.392671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b57ad-5606-42e8-bffc-7cf349f5a3d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:23.392975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b57ad-5606-42e8-bffc-7cf349f5a3d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:23.402636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b57ad-5606-42e8-bffc-7cf349f5a3d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:23.404090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b57ad-5606-42e8-bffc-7cf349f5a3d7 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:23.414301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 961/1918] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:24.113886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c47005-a4dc-4ad7-80e0-ad7572f930e8 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:24.115918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b5c47005-a4dc-4ad7-80e0-ad7572f930e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:24.127797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c47005-a4dc-4ad7-80e0-ad7572f930e8 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:24.132660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:24.133548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:24.133859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:24.134146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] PATH_INFO: `/v3/projects/6b1291d3b2234563bfb45357253e97a7` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:24.135345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b1291d3b2234563bfb45357253e97a7)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:24.137256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:24.262941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b1291d3b2234563bfb45357253e97a7', 'request_id': 'req-05cc7031-b717-41cc-bf7d-eaff361d68e6'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.269260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b1291d3b2234563bfb45357253e97a7', 'request_id': 'req-05cc7031-b717-41cc-bf7d-eaff361d68e6'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.346656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b1291d3b2234563bfb45357253e97a7', 'request_id': 'req-05cc7031-b717-41cc-bf7d-eaff361d68e6'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.358897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] The token cache is being invalidate because project 6b1291d3b2234563bfb45357253e97a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:24.362847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.362847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.362847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 958/1919] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:24 2022] DELETE /identity/v3/projects/6b1291d3b2234563bfb45357253e97a7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:24.365573 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b57ad-5606-42e8-bffc-7cf349f5a3d7 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:24.375281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b57ad-5606-42e8-bffc-7cf349f5a3d7 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:24.433066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:24.434125 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:24.434414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:24.434695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] PATH_INFO: `/v3/users/a9a7be01a1704140a9efe6f067048074` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:24.435916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a7be01a1704140a9efe6f067048074)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:24.437883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:24.549609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:24.660933 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a7be01a1704140a9efe6f067048074'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.736587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a7be01a1704140a9efe6f067048074'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.737176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a7be01a1704140a9efe6f067048074'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.744726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a7be01a1704140a9efe6f067048074'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.747711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ef4a7ba9-0932-43be-a019-62a59fc1cbdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a7be01a1704140a9efe6f067048074'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:24.763154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 962/1920] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:24 2022] DELETE /identity/v3/users/a9a7be01a1704140a9efe6f067048074 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:25.305661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:25.308163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:25.331258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc7031-b717-41cc-bf7d-eaff361d68e6 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:25.405479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:25.409375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:25.409375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:25.409375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] PATH_INFO: `/v3/projects/7e3f9f8abdbe46988d725678b154bea1` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:25.409375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3f9f8abdbe46988d725678b154bea1)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:25.410174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:25.546433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3f9f8abdbe46988d725678b154bea1', 'request_id': 'req-657213be-6da2-4c61-8899-dc13c0ca9596'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:25.549932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:25.557552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3f9f8abdbe46988d725678b154bea1', 'request_id': 'req-657213be-6da2-4c61-8899-dc13c0ca9596'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:25.645435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3f9f8abdbe46988d725678b154bea1', 'request_id': 'req-657213be-6da2-4c61-8899-dc13c0ca9596'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:25.657293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] The token cache is being invalidate because project 7e3f9f8abdbe46988d725678b154bea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:25.657584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:25.658371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-657213be-6da2-4c61-8899-dc13c0ca9596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:25.660157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 959/1921] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:25 2022] DELETE /identity/v3/projects/7e3f9f8abdbe46988d725678b154bea1 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:25.665329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e2c3d-f93b-4ef0-8c5e-6c617502c0a9 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:25.665865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e2c3d-f93b-4ef0-8c5e-6c617502c0a9 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:25.665968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e2c3d-f93b-4ef0-8c5e-6c617502c0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:25.685744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-9e8e2c3d-f93b-4ef0-8c5e-6c617502c0a9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:25.687014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9e8e2c3d-f93b-4ef0-8c5e-6c617502c0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:25.751595 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 963/1922] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:25.760028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d4e8b-8d93-4362-a491-365bac64d842 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:25.760323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d4e8b-8d93-4362-a491-365bac64d842 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:25.760607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d4e8b-8d93-4362-a491-365bac64d842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:25.779148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a43d4e8b-8d93-4362-a491-365bac64d842 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:25.780358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a43d4e8b-8d93-4362-a491-365bac64d842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:25.788807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 960/1923] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:25.792954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e2c3d-f93b-4ef0-8c5e-6c617502c0a9 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:25.804140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e2c3d-f93b-4ef0-8c5e-6c617502c0a9 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:25.808808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764c1c7-3876-44f1-9b7c-4996ff3b17e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:25.809617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c1c7-3876-44f1-9b7c-4996ff3b17e1 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:25.810828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c1c7-3876-44f1-9b7c-4996ff3b17e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:25.812216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c1c7-3876-44f1-9b7c-4996ff3b17e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:25.813303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764c1c7-3876-44f1-9b7c-4996ff3b17e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:56:25.813618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764c1c7-3876-44f1-9b7c-4996ff3b17e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:25.816386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764c1c7-3876-44f1-9b7c-4996ff3b17e1 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:25.839366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 964/1924] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:56:25.843702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d4e8b-8d93-4362-a491-365bac64d842 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:25.853190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d4e8b-8d93-4362-a491-365bac64d842 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:25.858155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50bcfb-00df-4bf0-812f-5f088eef0d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:25.858754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50bcfb-00df-4bf0-812f-5f088eef0d53 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:25.859032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50bcfb-00df-4bf0-812f-5f088eef0d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:25.859303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50bcfb-00df-4bf0-812f-5f088eef0d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:25.860118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50bcfb-00df-4bf0-812f-5f088eef0d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:25.862080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50bcfb-00df-4bf0-812f-5f088eef0d53 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:25.979227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 961/1925] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:25 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:25.983677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764c1c7-3876-44f1-9b7c-4996ff3b17e1 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:25.994864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764c1c7-3876-44f1-9b7c-4996ff3b17e1 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:26.060368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb23be3-c0cb-4186-81ed-68fcd0140c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:26.060503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb23be3-c0cb-4186-81ed-68fcd0140c45 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:26.060707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb23be3-c0cb-4186-81ed-68fcd0140c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:26.060990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb23be3-c0cb-4186-81ed-68fcd0140c45 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:26.061773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb23be3-c0cb-4186-81ed-68fcd0140c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:26.065044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb23be3-c0cb-4186-81ed-68fcd0140c45 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:26.172675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 965/1926] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:25 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:26.177019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50bcfb-00df-4bf0-812f-5f088eef0d53 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:26.188195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50bcfb-00df-4bf0-812f-5f088eef0d53 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:26.193437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09a409-f541-47a9-9a67-115ac8e8133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:26.194164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09a409-f541-47a9-9a67-115ac8e8133e admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:26.194480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09a409-f541-47a9-9a67-115ac8e8133e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:26.194762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09a409-f541-47a9-9a67-115ac8e8133e admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:26.195522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09a409-f541-47a9-9a67-115ac8e8133e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:26.197116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09a409-f541-47a9-9a67-115ac8e8133e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:26.214916 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 962/1927] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:26.215152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb23be3-c0cb-4186-81ed-68fcd0140c45 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:26.236695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb23be3-c0cb-4186-81ed-68fcd0140c45 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:26.243806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef8155-9cf1-4968-afa3-f29ea16c58dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:26.244596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef8155-9cf1-4968-afa3-f29ea16c58dc admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:26.244981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef8155-9cf1-4968-afa3-f29ea16c58dc admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:26.245863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef8155-9cf1-4968-afa3-f29ea16c58dc admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:26.246284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef8155-9cf1-4968-afa3-f29ea16c58dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:26.248047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef8155-9cf1-4968-afa3-f29ea16c58dc admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:26.262764 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 966/1928] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:26.264005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09a409-f541-47a9-9a67-115ac8e8133e admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:26.275367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09a409-f541-47a9-9a67-115ac8e8133e admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:26.280171 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e3349-c0fe-418f-8695-e92e6e4e7fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:26.281189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e3349-c0fe-418f-8695-e92e6e4e7fff admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:26.281468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e3349-c0fe-418f-8695-e92e6e4e7fff admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:26.282903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e3349-c0fe-418f-8695-e92e6e4e7fff admin admin] PATH_INFO: `/v3/projects/b31e85f4af554b3cb88f0da558d4107e/users/f9953bb3652f49639ce4c7ae628069c3/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:26.302355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e3349-c0fe-418f-8695-e92e6e4e7fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31e85f4af554b3cb88f0da558d4107e, user_id=f9953bb3652f49639ce4c7ae628069c3, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:26.305298 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e3349-c0fe-418f-8695-e92e6e4e7fff admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:26.383420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 963/1929] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:26 2022] PUT /identity/v3/projects/b31e85f4af554b3cb88f0da558d4107e/users/f9953bb3652f49639ce4c7ae628069c3/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:26.393680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31449d4f-38c0-4627-8272-4725cce19a95 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:26.394003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31449d4f-38c0-4627-8272-4725cce19a95 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:26.394283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31449d4f-38c0-4627-8272-4725cce19a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:26.413395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:26.423101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-31449d4f-38c0-4627-8272-4725cce19a95 None None] MFA Rules not processed for user `f9953bb3652f49639ce4c7ae628069c3`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:26.424711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-31449d4f-38c0-4627-8272-4725cce19a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:26.563319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 967/1930] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:26.563704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e3349-c0fe-418f-8695-e92e6e4e7fff admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:26.699414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e3349-c0fe-418f-8695-e92e6e4e7fff admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:26.705382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea114e5-e2c6-4ba9-a5fd-58f606e133b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:26.706271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea114e5-e2c6-4ba9-a5fd-58f606e133b7 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:26.706684 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea114e5-e2c6-4ba9-a5fd-58f606e133b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:26.706811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea114e5-e2c6-4ba9-a5fd-58f606e133b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:26.708653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea114e5-e2c6-4ba9-a5fd-58f606e133b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:26.710716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea114e5-e2c6-4ba9-a5fd-58f606e133b7 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:26.870727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 964/1931] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:26 2022] POST /identity/v3/projects => generated 383 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:26.874494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31449d4f-38c0-4627-8272-4725cce19a95 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:26.894815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31449d4f-38c0-4627-8272-4725cce19a95 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:26.953570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf6540-d76d-46e0-9557-97db26fa0add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:26.954295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf6540-d76d-46e0-9557-97db26fa0add admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:26.954582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf6540-d76d-46e0-9557-97db26fa0add admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:26.954861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf6540-d76d-46e0-9557-97db26fa0add admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:26.955646 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf6540-d76d-46e0-9557-97db26fa0add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:26.957627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf6540-d76d-46e0-9557-97db26fa0add admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:27.107042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 968/1932] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:26 2022] POST /identity/v3/users => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:27.107142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea114e5-e2c6-4ba9-a5fd-58f606e133b7 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:27.118767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea114e5-e2c6-4ba9-a5fd-58f606e133b7 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:27.124508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b4040-6ef3-4d67-968a-b7027f34b2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:27.125231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b4040-6ef3-4d67-968a-b7027f34b2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.125514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b4040-6ef3-4d67-968a-b7027f34b2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.125790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b4040-6ef3-4d67-968a-b7027f34b2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.126559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b4040-6ef3-4d67-968a-b7027f34b2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:27.128402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b4040-6ef3-4d67-968a-b7027f34b2d5 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:27.140699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 965/1933] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:27.144962 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf6540-d76d-46e0-9557-97db26fa0add admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:27.155918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf6540-d76d-46e0-9557-97db26fa0add admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:27.165755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104631d-bdaa-4297-8f7d-3765b92f1d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:27.167212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104631d-bdaa-4297-8f7d-3765b92f1d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.167675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104631d-bdaa-4297-8f7d-3765b92f1d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.169615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104631d-bdaa-4297-8f7d-3765b92f1d9d admin admin] PATH_INFO: `/v3/projects/d3b4b6b862524b75bd6eef07a85ecc5f/users/56628a93fcc64853b1c2647ad964c89f/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.192497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104631d-bdaa-4297-8f7d-3765b92f1d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3b4b6b862524b75bd6eef07a85ecc5f, user_id=56628a93fcc64853b1c2647ad964c89f, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:27.196425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104631d-bdaa-4297-8f7d-3765b92f1d9d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:27.289492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 969/1934] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:27 2022] PUT /identity/v3/projects/d3b4b6b862524b75bd6eef07a85ecc5f/users/56628a93fcc64853b1c2647ad964c89f/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:27.291601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b4040-6ef3-4d67-968a-b7027f34b2d5 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:27.310827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b4040-6ef3-4d67-968a-b7027f34b2d5 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:27.365008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:27.380507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf7714-eb95-4775-8fb9-f9c1412d4809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:27.380507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7714-eb95-4775-8fb9-f9c1412d4809 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.380507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7714-eb95-4775-8fb9-f9c1412d4809 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.380507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7714-eb95-4775-8fb9-f9c1412d4809 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.381060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf7714-eb95-4775-8fb9-f9c1412d4809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:27.381060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf7714-eb95-4775-8fb9-f9c1412d4809 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:27.390378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 966/1935] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:27.394372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104631d-bdaa-4297-8f7d-3765b92f1d9d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:27.404984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104631d-bdaa-4297-8f7d-3765b92f1d9d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:27.410259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff1f4b-c8bf-449d-945f-7dc13e1ba037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:27.411325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff1f4b-c8bf-449d-945f-7dc13e1ba037 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.411668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff1f4b-c8bf-449d-945f-7dc13e1ba037 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.412485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff1f4b-c8bf-449d-945f-7dc13e1ba037 admin admin] PATH_INFO: `/v3/projects/d3b4b6b862524b75bd6eef07a85ecc5f/users/56628a93fcc64853b1c2647ad964c89f/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.414499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff1f4b-c8bf-449d-945f-7dc13e1ba037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3b4b6b862524b75bd6eef07a85ecc5f, user_id=56628a93fcc64853b1c2647ad964c89f, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:27.417145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff1f4b-c8bf-449d-945f-7dc13e1ba037 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:27.429628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 970/1936] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:27 2022] PUT /identity/v3/projects/d3b4b6b862524b75bd6eef07a85ecc5f/users/56628a93fcc64853b1c2647ad964c89f/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:27.443030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97044d4-f527-4481-9584-f1d4bcf09c38 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.447070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97044d4-f527-4481-9584-f1d4bcf09c38 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.447070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97044d4-f527-4481-9584-f1d4bcf09c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.456213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e105aa-8b0f-4da4-87c3-ec92db6d22a9 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.456562 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e105aa-8b0f-4da4-87c3-ec92db6d22a9 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.456852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e105aa-8b0f-4da4-87c3-ec92db6d22a9 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.458332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 971/1937] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 15:56:27.468889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c34ba-ed86-4c37-88d8-2d7957ee095d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.469363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c34ba-ed86-4c37-88d8-2d7957ee095d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.469714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c34ba-ed86-4c37-88d8-2d7957ee095d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.471884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f97044d4-f527-4481-9584-f1d4bcf09c38 None None] MFA Rules not processed for user `56628a93fcc64853b1c2647ad964c89f`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:27.473103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f97044d4-f527-4481-9584-f1d4bcf09c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:27.502223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-604c34ba-ed86-4c37-88d8-2d7957ee095d None None] MFA Rules not processed for user `97524081e59f4775b83d82edf737810e`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:27.503399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-604c34ba-ed86-4c37-88d8-2d7957ee095d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:27.551115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:27.556607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 967/1938] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:27.568425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97044d4-f527-4481-9584-f1d4bcf09c38 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:27.569968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f97044d4-f527-4481-9584-f1d4bcf09c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:27.574449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 972/1939] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:56:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:27.586228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c34ba-ed86-4c37-88d8-2d7957ee095d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:27.587633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-604c34ba-ed86-4c37-88d8-2d7957ee095d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:27.601027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c34ba-ed86-4c37-88d8-2d7957ee095d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:27.607048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb116cd-3db5-4572-8134-a767fd6825f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:27.607728 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb116cd-3db5-4572-8134-a767fd6825f6 service placement] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.608012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb116cd-3db5-4572-8134-a767fd6825f6 service placement] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.608288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb116cd-3db5-4572-8134-a767fd6825f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.621072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb116cd-3db5-4572-8134-a767fd6825f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:27.623226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb116cd-3db5-4572-8134-a767fd6825f6 service placement] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:27.633330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 973/1940] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:27.634324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb116cd-3db5-4572-8134-a767fd6825f6 service placement] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:27.635559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2cb116cd-3db5-4572-8134-a767fd6825f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:27.649313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97044d4-f527-4481-9584-f1d4bcf09c38 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:27.653181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf2a52-4435-498b-b5c9-935c7d1f446e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:27.653889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf2a52-4435-498b-b5c9-935c7d1f446e service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.656786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf2a52-4435-498b-b5c9-935c7d1f446e service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.656786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf2a52-4435-498b-b5c9-935c7d1f446e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.664757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf2a52-4435-498b-b5c9-935c7d1f446e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:27.666326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf2a52-4435-498b-b5c9-935c7d1f446e service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:27.676538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 968/1941] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:27.702919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb116cd-3db5-4572-8134-a767fd6825f6 service placement] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:27.711477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:27.712495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.712816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.713090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] PATH_INFO: `/v3/users/d476a2bec8164b8bbf9a5aaa419a0351` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.714343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d476a2bec8164b8bbf9a5aaa419a0351)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:27.716258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:27.813586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf2a52-4435-498b-b5c9-935c7d1f446e service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:27.823022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf2a52-4435-498b-b5c9-935c7d1f446e service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:27.826554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b04eff-4dec-4410-aed7-cb34c9333215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:27.827189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b04eff-4dec-4410-aed7-cb34c9333215 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:27.827510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b04eff-4dec-4410-aed7-cb34c9333215 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:27.827786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b04eff-4dec-4410-aed7-cb34c9333215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:27.886876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b04eff-4dec-4410-aed7-cb34c9333215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:27.888314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b04eff-4dec-4410-aed7-cb34c9333215 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:27.897907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 969/1942] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:27.931051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd476a2bec8164b8bbf9a5aaa419a0351'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.003634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd476a2bec8164b8bbf9a5aaa419a0351'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.004378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd476a2bec8164b8bbf9a5aaa419a0351'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.012025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd476a2bec8164b8bbf9a5aaa419a0351'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.012684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd476a2bec8164b8bbf9a5aaa419a0351'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.053880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 974/1943] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:27 2022] DELETE /identity/v3/users/d476a2bec8164b8bbf9a5aaa419a0351 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:28.315393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b04eff-4dec-4410-aed7-cb34c9333215 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:28.326658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b04eff-4dec-4410-aed7-cb34c9333215 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:28.336487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:28.384616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:28.385408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:28.385696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:28.385993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] PATH_INFO: `/v3/projects/73dcc59c5829442da0518976525fe40b` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:28.387178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73dcc59c5829442da0518976525fe40b)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:28.389082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:28.408800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:28.410486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:28.424278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d138c-c419-44fd-ae08-b73a81e9e1e3 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:28.434077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:28.434077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:28.434077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:28.434077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] PATH_INFO: `/v3/users/f9953bb3652f49639ce4c7ae628069c3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:28.434457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9953bb3652f49639ce4c7ae628069c3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:28.434457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:28.523640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73dcc59c5829442da0518976525fe40b', 'request_id': 'req-59501826-f6c1-4687-ac20-a90d9c4fb022'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.530271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73dcc59c5829442da0518976525fe40b', 'request_id': 'req-59501826-f6c1-4687-ac20-a90d9c4fb022'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.599186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73dcc59c5829442da0518976525fe40b', 'request_id': 'req-59501826-f6c1-4687-ac20-a90d9c4fb022'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.611040 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] The token cache is being invalidate because project 73dcc59c5829442da0518976525fe40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:28.611321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.611961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-59501826-f6c1-4687-ac20-a90d9c4fb022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.613498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 970/1944] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:28 2022] DELETE /identity/v3/projects/73dcc59c5829442da0518976525fe40b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:28.618825 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb69736-d399-429d-ba79-d350ff95f96e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:28.619112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb69736-d399-429d-ba79-d350ff95f96e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:28.619392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb69736-d399-429d-ba79-d350ff95f96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:28.638030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-dcb69736-d399-429d-ba79-d350ff95f96e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:28.639215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-dcb69736-d399-429d-ba79-d350ff95f96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:28.660946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9953bb3652f49639ce4c7ae628069c3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.663332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:28.699831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 971/1945] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:28.708517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72d66f-c97e-4831-a2be-08b7dc6cf584 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:28.708838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72d66f-c97e-4831-a2be-08b7dc6cf584 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:28.709122 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72d66f-c97e-4831-a2be-08b7dc6cf584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:28.727054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-2c72d66f-c97e-4831-a2be-08b7dc6cf584 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:28.728258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2c72d66f-c97e-4831-a2be-08b7dc6cf584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:28.736101 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 972/1946] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:28.740576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72d66f-c97e-4831-a2be-08b7dc6cf584 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:28.748016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9953bb3652f49639ce4c7ae628069c3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.748634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9953bb3652f49639ce4c7ae628069c3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.749298 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72d66f-c97e-4831-a2be-08b7dc6cf584 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:28.754112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:28.754775 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:28.755090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:28.755371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:28.756129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:56:28.756355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9953bb3652f49639ce4c7ae628069c3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.756452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:28.756988 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20e9f030-f343-42ef-beba-39c980592b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9953bb3652f49639ce4c7ae628069c3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:28.758083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:28.770132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 973/1947] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:56:28.774289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:28.775997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:28.778356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 975/1948] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:28 2022] DELETE /identity/v3/users/f9953bb3652f49639ce4c7ae628069c3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:28.782820 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3c806-3c49-4740-b94c-c83c794bcb42 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:28.783107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3c806-3c49-4740-b94c-c83c794bcb42 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:28.783383 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3c806-3c49-4740-b94c-c83c794bcb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:28.789856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb07d8-e6b8-42b2-b2e3-eb3249caa274 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:28.802198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-48d3c806-3c49-4740-b94c-c83c794bcb42 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:28.804036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-48d3c806-3c49-4740-b94c-c83c794bcb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:28.811993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:28.847719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed3636-6cd5-43f1-a524-cda6029f96d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:28.848426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed3636-6cd5-43f1-a524-cda6029f96d3 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:28.848711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed3636-6cd5-43f1-a524-cda6029f96d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:28.849016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed3636-6cd5-43f1-a524-cda6029f96d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:28.850542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed3636-6cd5-43f1-a524-cda6029f96d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:28.852485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed3636-6cd5-43f1-a524-cda6029f96d3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:28.864744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 976/1949] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:28.874445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3c806-3c49-4740-b94c-c83c794bcb42 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:28.875802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-48d3c806-3c49-4740-b94c-c83c794bcb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:28.932637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3c806-3c49-4740-b94c-c83c794bcb42 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:28.937188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0974655-46c3-45b7-a6ae-0ee7a53ea324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:28.943785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0974655-46c3-45b7-a6ae-0ee7a53ea324 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:28.943785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0974655-46c3-45b7-a6ae-0ee7a53ea324 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:28.943785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0974655-46c3-45b7-a6ae-0ee7a53ea324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:28.944095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 974/1950] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:28 2022] POST /identity/v3/projects => generated 405 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:28.944095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed3636-6cd5-43f1-a524-cda6029f96d3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:28.952600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0974655-46c3-45b7-a6ae-0ee7a53ea324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:28.954083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0974655-46c3-45b7-a6ae-0ee7a53ea324 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:28.965308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 977/1951] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:29.007341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed3636-6cd5-43f1-a524-cda6029f96d3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.012370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61026b3-d518-424f-9af0-6c0e9086541a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.013189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61026b3-d518-424f-9af0-6c0e9086541a admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.013477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61026b3-d518-424f-9af0-6c0e9086541a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.013759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61026b3-d518-424f-9af0-6c0e9086541a admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.014515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61026b3-d518-424f-9af0-6c0e9086541a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.016472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61026b3-d518-424f-9af0-6c0e9086541a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:29.099886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 975/1952] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:28 2022] POST /identity/v3/users => generated 303 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:29.103915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0974655-46c3-45b7-a6ae-0ee7a53ea324 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:29.114142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0974655-46c3-45b7-a6ae-0ee7a53ea324 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.119651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f6394-c38e-49ed-acb2-b06aea132b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.120324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f6394-c38e-49ed-acb2-b06aea132b0c admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.120614 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f6394-c38e-49ed-acb2-b06aea132b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.120893 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f6394-c38e-49ed-acb2-b06aea132b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.121625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f6394-c38e-49ed-acb2-b06aea132b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.123291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f6394-c38e-49ed-acb2-b06aea132b0c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:29.137641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 978/1953] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:29.138946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61026b3-d518-424f-9af0-6c0e9086541a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:29.149143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61026b3-d518-424f-9af0-6c0e9086541a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.153908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48064382-1e8b-4b11-9a36-6d3847e453b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.154646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064382-1e8b-4b11-9a36-6d3847e453b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.154865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064382-1e8b-4b11-9a36-6d3847e453b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.155147 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064382-1e8b-4b11-9a36-6d3847e453b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.155880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48064382-1e8b-4b11-9a36-6d3847e453b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.157475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48064382-1e8b-4b11-9a36-6d3847e453b1 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:29.172003 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 976/1954] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:29.173589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f6394-c38e-49ed-acb2-b06aea132b0c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:29.185976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f6394-c38e-49ed-acb2-b06aea132b0c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.191257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b592c-966b-47a0-9407-e8ec53589730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.192168 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b592c-966b-47a0-9407-e8ec53589730 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.192453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b592c-966b-47a0-9407-e8ec53589730 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.193011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b592c-966b-47a0-9407-e8ec53589730 admin admin] PATH_INFO: `/v3/projects/38935f70cd874a21b77f496796d3e942/users/51040eebe8f4487c9d47232085ab2d87/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.228716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b592c-966b-47a0-9407-e8ec53589730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38935f70cd874a21b77f496796d3e942, user_id=51040eebe8f4487c9d47232085ab2d87, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.231597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b592c-966b-47a0-9407-e8ec53589730 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:29.340760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 979/1955] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:29 2022] PUT /identity/v3/projects/38935f70cd874a21b77f496796d3e942/users/51040eebe8f4487c9d47232085ab2d87/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:29.351061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea1027-82f2-4d39-bcd1-0c2adfa43dc1 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.351353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea1027-82f2-4d39-bcd1-0c2adfa43dc1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.351637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea1027-82f2-4d39-bcd1-0c2adfa43dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.378194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-44ea1027-82f2-4d39-bcd1-0c2adfa43dc1 None None] MFA Rules not processed for user `51040eebe8f4487c9d47232085ab2d87`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:29.379405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-44ea1027-82f2-4d39-bcd1-0c2adfa43dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:29.446387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 977/1956] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:29.450866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b592c-966b-47a0-9407-e8ec53589730 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:29.490191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:29.491316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea1027-82f2-4d39-bcd1-0c2adfa43dc1 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:29.492963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-44ea1027-82f2-4d39-bcd1-0c2adfa43dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:29.514442 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b592c-966b-47a0-9407-e8ec53589730 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.515260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:29.519336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97970bb-3bf5-4695-a323-c7e87e5ea938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.519886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97970bb-3bf5-4695-a323-c7e87e5ea938 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.520181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97970bb-3bf5-4695-a323-c7e87e5ea938 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.520462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97970bb-3bf5-4695-a323-c7e87e5ea938 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.521926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97970bb-3bf5-4695-a323-c7e87e5ea938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.523870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97970bb-3bf5-4695-a323-c7e87e5ea938 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:29.561211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea1027-82f2-4d39-bcd1-0c2adfa43dc1 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.566175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.567155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.567447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.567727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] PATH_INFO: `/v3/projects/b31e85f4af554b3cb88f0da558d4107e` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.569556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31e85f4af554b3cb88f0da558d4107e)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.571579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:29.653573 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 980/1957] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:29 2022] POST /identity/v3/projects => generated 405 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:29.657467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97970bb-3bf5-4695-a323-c7e87e5ea938 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:29.709371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31e85f4af554b3cb88f0da558d4107e', 'request_id': 'req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:29.714918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31e85f4af554b3cb88f0da558d4107e', 'request_id': 'req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:29.719046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97970bb-3bf5-4695-a323-c7e87e5ea938 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.774983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd783420-1ad3-4108-a32d-39b66605895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.775793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd783420-1ad3-4108-a32d-39b66605895d admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.776078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd783420-1ad3-4108-a32d-39b66605895d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.776361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd783420-1ad3-4108-a32d-39b66605895d admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.776666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31e85f4af554b3cb88f0da558d4107e', 'request_id': 'req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:29.777056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd783420-1ad3-4108-a32d-39b66605895d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.779079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd783420-1ad3-4108-a32d-39b66605895d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:29.788424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] The token cache is being invalidate because project b31e85f4af554b3cb88f0da558d4107e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:29.788803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:29.789560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:29.791269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 978/1958] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:29 2022] DELETE /identity/v3/projects/b31e85f4af554b3cb88f0da558d4107e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:29.795092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:29.796374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:29.809775 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbad1d2-6d79-43b7-ba29-cdbafb41647d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.814424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.815325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.815606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.815883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] PATH_INFO: `/v3/users/56628a93fcc64853b1c2647ad964c89f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.817066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56628a93fcc64853b1c2647ad964c89f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.819656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:29.885901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 981/1959] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:29 2022] POST /identity/v3/users => generated 309 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:29.889623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd783420-1ad3-4108-a32d-39b66605895d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:29.891249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fd783420-1ad3-4108-a32d-39b66605895d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:29.902814 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd783420-1ad3-4108-a32d-39b66605895d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.907417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcfacb-4a83-4fcf-ab08-0a03d86775b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.908056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcfacb-4a83-4fcf-ab08-0a03d86775b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.908345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcfacb-4a83-4fcf-ab08-0a03d86775b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.908626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcfacb-4a83-4fcf-ab08-0a03d86775b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.909349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcfacb-4a83-4fcf-ab08-0a03d86775b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.910993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bcfacb-4a83-4fcf-ab08-0a03d86775b1 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:29.921124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 982/1960] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:56:29.925071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcfacb-4a83-4fcf-ab08-0a03d86775b1 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:29.935496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcfacb-4a83-4fcf-ab08-0a03d86775b1 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:29.940674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261981-be01-441c-9252-f3a5133c8b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:29.941558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261981-be01-441c-9252-f3a5133c8b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:29.941941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261981-be01-441c-9252-f3a5133c8b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:29.942229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261981-be01-441c-9252-f3a5133c8b81 admin admin] PATH_INFO: `/v3/projects/1c5b55f3ee3d45c8a020595b52c18eff/users/f5650a6cf89a46e5b088d5b49a1969e6/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:29.958392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57261981-be01-441c-9252-f3a5133c8b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5b55f3ee3d45c8a020595b52c18eff, user_id=f5650a6cf89a46e5b088d5b49a1969e6, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:29.961202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57261981-be01-441c-9252-f3a5133c8b81 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:30.080945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 983/1961] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:29 2022] PUT /identity/v3/projects/1c5b55f3ee3d45c8a020595b52c18eff/users/f5650a6cf89a46e5b088d5b49a1969e6/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:30.084975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261981-be01-441c-9252-f3a5133c8b81 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:30.088376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56628a93fcc64853b1c2647ad964c89f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.147631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261981-be01-441c-9252-f3a5133c8b81 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:30.152412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14384b-63d9-4fdc-822a-24edfcc8035c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:30.153096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14384b-63d9-4fdc-822a-24edfcc8035c admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:30.153381 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14384b-63d9-4fdc-822a-24edfcc8035c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:30.153659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14384b-63d9-4fdc-822a-24edfcc8035c admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:30.154438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14384b-63d9-4fdc-822a-24edfcc8035c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:30.156116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14384b-63d9-4fdc-822a-24edfcc8035c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:30.166731 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 984/1962] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:30.171009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14384b-63d9-4fdc-822a-24edfcc8035c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:30.181784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14384b-63d9-4fdc-822a-24edfcc8035c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:30.186610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364d037-acfd-40d7-b737-862f14538616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:30.187509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364d037-acfd-40d7-b737-862f14538616 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:30.187799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364d037-acfd-40d7-b737-862f14538616 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:30.188077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364d037-acfd-40d7-b737-862f14538616 admin admin] PATH_INFO: `/v3/projects/1c5b55f3ee3d45c8a020595b52c18eff/users/f5650a6cf89a46e5b088d5b49a1969e6/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:30.188472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56628a93fcc64853b1c2647ad964c89f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.189152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56628a93fcc64853b1c2647ad964c89f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.190283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364d037-acfd-40d7-b737-862f14538616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5b55f3ee3d45c8a020595b52c18eff, user_id=f5650a6cf89a46e5b088d5b49a1969e6, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:30.193221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364d037-acfd-40d7-b737-862f14538616 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:30.197622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56628a93fcc64853b1c2647ad964c89f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.201759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56628a93fcc64853b1c2647ad964c89f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.204522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 985/1963] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:30 2022] PUT /identity/v3/projects/1c5b55f3ee3d45c8a020595b52c18eff/users/f5650a6cf89a46e5b088d5b49a1969e6/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:30.214774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c4143-a761-41cc-91c5-ba37fba2b9a8 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:30.215094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c4143-a761-41cc-91c5-ba37fba2b9a8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:30.215345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c4143-a761-41cc-91c5-ba37fba2b9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:30.217933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 979/1964] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:29 2022] DELETE /identity/v3/users/56628a93fcc64853b1c2647ad964c89f => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:30.242783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ab1c4143-a761-41cc-91c5-ba37fba2b9a8 None None] MFA Rules not processed for user `f5650a6cf89a46e5b088d5b49a1969e6`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:30.244111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ab1c4143-a761-41cc-91c5-ba37fba2b9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:30.251173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:30.322162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 986/1965] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:30.332691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:30.334698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:30.356210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:30.394161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0ba2f-0742-45e9-b9cf-847ca37ae329 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:30.397882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dad231-e89f-4596-99e5-2ece12c40af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:30.398544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dad231-e89f-4596-99e5-2ece12c40af6 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:30.398837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dad231-e89f-4596-99e5-2ece12c40af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:30.399113 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dad231-e89f-4596-99e5-2ece12c40af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:30.411386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dad231-e89f-4596-99e5-2ece12c40af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:30.413565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dad231-e89f-4596-99e5-2ece12c40af6 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:30.427905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 980/1966] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:30 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:30.580816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c4143-a761-41cc-91c5-ba37fba2b9a8 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:30.582256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ab1c4143-a761-41cc-91c5-ba37fba2b9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:30.647496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c4143-a761-41cc-91c5-ba37fba2b9a8 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:30.652194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:30.653123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:30.653405 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:30.653684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] PATH_INFO: `/v3/users/51040eebe8f4487c9d47232085ab2d87` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:30.654938 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51040eebe8f4487c9d47232085ab2d87)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:30.656970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:30.686974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dad231-e89f-4596-99e5-2ece12c40af6 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:30.688473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-16dad231-e89f-4596-99e5-2ece12c40af6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:30.705548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dad231-e89f-4596-99e5-2ece12c40af6 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:30.713041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:30.713041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:30.713041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:30.713472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] PATH_INFO: `/v3/projects/d3b4b6b862524b75bd6eef07a85ecc5f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:30.714505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3b4b6b862524b75bd6eef07a85ecc5f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:30.716415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:30.857782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:30.870860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51040eebe8f4487c9d47232085ab2d87'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.871199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3b4b6b862524b75bd6eef07a85ecc5f', 'request_id': 'req-6f32b45d-1579-4e51-8709-aec130303153'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.877088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3b4b6b862524b75bd6eef07a85ecc5f', 'request_id': 'req-6f32b45d-1579-4e51-8709-aec130303153'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.987481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51040eebe8f4487c9d47232085ab2d87'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.988165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51040eebe8f4487c9d47232085ab2d87'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.995694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51040eebe8f4487c9d47232085ab2d87'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:30.996393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6dcc7616-0c11-4650-822d-d0440069a945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51040eebe8f4487c9d47232085ab2d87'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:31.014543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 987/1967] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:30 2022] DELETE /identity/v3/users/51040eebe8f4487c9d47232085ab2d87 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:31.018827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3efee-5676-4c52-b9fd-f50e715b72b1 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.019123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3efee-5676-4c52-b9fd-f50e715b72b1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.019404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3efee-5676-4c52-b9fd-f50e715b72b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.036864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-79b3efee-5676-4c52-b9fd-f50e715b72b1 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:31.038114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-79b3efee-5676-4c52-b9fd-f50e715b72b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:31.051272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3b4b6b862524b75bd6eef07a85ecc5f', 'request_id': 'req-6f32b45d-1579-4e51-8709-aec130303153'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:31.063525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] The token cache is being invalidate because project d3b4b6b862524b75bd6eef07a85ecc5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:31.063816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:31.064464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-6f32b45d-1579-4e51-8709-aec130303153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:31.066093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 981/1968] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:30 2022] DELETE /identity/v3/projects/d3b4b6b862524b75bd6eef07a85ecc5f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:31.071367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b437acf-ef8a-4174-b328-72976d984679 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.071653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b437acf-ef8a-4174-b328-72976d984679 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.071990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b437acf-ef8a-4174-b328-72976d984679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.094143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8b437acf-ef8a-4174-b328-72976d984679 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:31.095441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8b437acf-ef8a-4174-b328-72976d984679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:31.100320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 988/1969] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:31.105859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 982/1970] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:31.110195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3efee-5676-4c52-b9fd-f50e715b72b1 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.111111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5e5c4-61ad-4995-b52e-884028ecff28 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.111519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-79b3efee-5676-4c52-b9fd-f50e715b72b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:31.111625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5e5c4-61ad-4995-b52e-884028ecff28 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.111832 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5e5c4-61ad-4995-b52e-884028ecff28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.131061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-38b5e5c4-61ad-4995-b52e-884028ecff28 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:31.132538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-38b5e5c4-61ad-4995-b52e-884028ecff28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:31.142765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 983/1971] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:31.147458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5e5c4-61ad-4995-b52e-884028ecff28 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.156975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5e5c4-61ad-4995-b52e-884028ecff28 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.161631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ee737-713d-4c93-8c6c-5920e49dd131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.162336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ee737-713d-4c93-8c6c-5920e49dd131 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.162622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ee737-713d-4c93-8c6c-5920e49dd131 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.162904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ee737-713d-4c93-8c6c-5920e49dd131 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.163714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ee737-713d-4c93-8c6c-5920e49dd131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:56:31.164026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ee737-713d-4c93-8c6c-5920e49dd131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.165747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ee737-713d-4c93-8c6c-5920e49dd131 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:31.169622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3efee-5676-4c52-b9fd-f50e715b72b1 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.173416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a50b0b-5250-4f7a-a965-ca5499377104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.176238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50b0b-5250-4f7a-a965-ca5499377104 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.176238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50b0b-5250-4f7a-a965-ca5499377104 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.176238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50b0b-5250-4f7a-a965-ca5499377104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.177754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 984/1972] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:56:31.181983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ee737-713d-4c93-8c6c-5920e49dd131 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.184704 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a50b0b-5250-4f7a-a965-ca5499377104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.186172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a50b0b-5250-4f7a-a965-ca5499377104 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:31.191743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ee737-713d-4c93-8c6c-5920e49dd131 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.196982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 989/1973] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:31.197784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a378a7-39d1-42b9-8df9-42ebaeafbd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.198644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a378a7-39d1-42b9-8df9-42ebaeafbd94 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.198936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a378a7-39d1-42b9-8df9-42ebaeafbd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.199323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a378a7-39d1-42b9-8df9-42ebaeafbd94 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.200081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a378a7-39d1-42b9-8df9-42ebaeafbd94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.203118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a378a7-39d1-42b9-8df9-42ebaeafbd94 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:31.298326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 985/1974] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:31 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:31.302703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a50b0b-5250-4f7a-a965-ca5499377104 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.312263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a50b0b-5250-4f7a-a965-ca5499377104 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.361174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:31.370750 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148c526-6e51-4c1e-affd-36663791f735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.371420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148c526-6e51-4c1e-affd-36663791f735 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.371706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148c526-6e51-4c1e-affd-36663791f735 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.371982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148c526-6e51-4c1e-affd-36663791f735 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.372683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148c526-6e51-4c1e-affd-36663791f735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.374647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148c526-6e51-4c1e-affd-36663791f735 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:31.469849 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 990/1975] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:31 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:31.473341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a378a7-39d1-42b9-8df9-42ebaeafbd94 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.501989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a378a7-39d1-42b9-8df9-42ebaeafbd94 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.508757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37124820-12f3-4afb-b9a7-b9f646a2aa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.509449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37124820-12f3-4afb-b9a7-b9f646a2aa5d admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.509939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37124820-12f3-4afb-b9a7-b9f646a2aa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.510220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37124820-12f3-4afb-b9a7-b9f646a2aa5d admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.511140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37124820-12f3-4afb-b9a7-b9f646a2aa5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.512904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37124820-12f3-4afb-b9a7-b9f646a2aa5d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:31.527315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 986/1976] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:31.528602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148c526-6e51-4c1e-affd-36663791f735 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.540174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148c526-6e51-4c1e-affd-36663791f735 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.545207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cb3a0-010a-4f43-879c-9408311eab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.545953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cb3a0-010a-4f43-879c-9408311eab32 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.546266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cb3a0-010a-4f43-879c-9408311eab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.546545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cb3a0-010a-4f43-879c-9408311eab32 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.547319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cb3a0-010a-4f43-879c-9408311eab32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.549008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cb3a0-010a-4f43-879c-9408311eab32 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:31.563548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 991/1977] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:31.564864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37124820-12f3-4afb-b9a7-b9f646a2aa5d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.575557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37124820-12f3-4afb-b9a7-b9f646a2aa5d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.580139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47987af-cb33-4d4f-89c5-e7baf76fde0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.581036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47987af-cb33-4d4f-89c5-e7baf76fde0e admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.581320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47987af-cb33-4d4f-89c5-e7baf76fde0e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.581606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47987af-cb33-4d4f-89c5-e7baf76fde0e admin admin] PATH_INFO: `/v3/projects/28ea3dc3799b4c3591d516d6c65cb935/users/2ce6c880e4274e97ac1f24bd92c48fc9/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.597582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47987af-cb33-4d4f-89c5-e7baf76fde0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=28ea3dc3799b4c3591d516d6c65cb935, user_id=2ce6c880e4274e97ac1f24bd92c48fc9, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.600620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47987af-cb33-4d4f-89c5-e7baf76fde0e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:31.682070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cb3a0-010a-4f43-879c-9408311eab32 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.683855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-064cb3a0-010a-4f43-879c-9408311eab32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:31.697117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cb3a0-010a-4f43-879c-9408311eab32 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.701839 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.702817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.703002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.703274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] PATH_INFO: `/v3/projects/38935f70cd874a21b77f496796d3e942` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.704465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38935f70cd874a21b77f496796d3e942)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.706444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:31.709261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 987/1978] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:31 2022] PUT /identity/v3/projects/28ea3dc3799b4c3591d516d6c65cb935/users/2ce6c880e4274e97ac1f24bd92c48fc9/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:31.720559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f06ed-1b57-4b55-aa8b-a602f5db3fa2 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.720867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f06ed-1b57-4b55-aa8b-a602f5db3fa2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.721131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1f06ed-1b57-4b55-aa8b-a602f5db3fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.747939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-1a1f06ed-1b57-4b55-aa8b-a602f5db3fa2 None None] MFA Rules not processed for user `2ce6c880e4274e97ac1f24bd92c48fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:31.749296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1a1f06ed-1b57-4b55-aa8b-a602f5db3fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:31.758380 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:31.811651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 988/1979] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:31.816144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f06ed-1b57-4b55-aa8b-a602f5db3fa2 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.835461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38935f70cd874a21b77f496796d3e942', 'request_id': 'req-30d1480d-b72b-408c-a332-4817fa4f0f66'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:31.841412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38935f70cd874a21b77f496796d3e942', 'request_id': 'req-30d1480d-b72b-408c-a332-4817fa4f0f66'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:31.877774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1f06ed-1b57-4b55-aa8b-a602f5db3fa2 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.932983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d197ca2-0eb7-471b-98ab-92d07a49f557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.933582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d197ca2-0eb7-471b-98ab-92d07a49f557 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.933895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d197ca2-0eb7-471b-98ab-92d07a49f557 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.934177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d197ca2-0eb7-471b-98ab-92d07a49f557 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.934872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d197ca2-0eb7-471b-98ab-92d07a49f557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.936777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d197ca2-0eb7-471b-98ab-92d07a49f557 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:31.953404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38935f70cd874a21b77f496796d3e942', 'request_id': 'req-30d1480d-b72b-408c-a332-4817fa4f0f66'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:31.966961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] The token cache is being invalidate because project 38935f70cd874a21b77f496796d3e942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:31.967253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:31.967912 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:31.969464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 992/1980] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:31 2022] DELETE /identity/v3/projects/38935f70cd874a21b77f496796d3e942 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:31.973322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:31.975201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:31.987690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1480d-b72b-408c-a332-4817fa4f0f66 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:31.992324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:31.993179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:31.994264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:31.994264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] PATH_INFO: `/v3/users/f5650a6cf89a46e5b088d5b49a1969e6` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:31.995055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5650a6cf89a46e5b088d5b49a1969e6)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:31.996969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:32.018232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 989/1981] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:31 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:32.022850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d197ca2-0eb7-471b-98ab-92d07a49f557 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:32.024448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2d197ca2-0eb7-471b-98ab-92d07a49f557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:32.087283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d197ca2-0eb7-471b-98ab-92d07a49f557 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:32.092727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f63c9-38f8-4f6a-bd4e-a98da0273d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:32.093406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f63c9-38f8-4f6a-bd4e-a98da0273d09 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.093717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f63c9-38f8-4f6a-bd4e-a98da0273d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.094027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f63c9-38f8-4f6a-bd4e-a98da0273d09 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.094714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f63c9-38f8-4f6a-bd4e-a98da0273d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:32.096658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f63c9-38f8-4f6a-bd4e-a98da0273d09 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:32.194629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 990/1982] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:32 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:32.198434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f63c9-38f8-4f6a-bd4e-a98da0273d09 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:32.208753 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f63c9-38f8-4f6a-bd4e-a98da0273d09 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:32.213350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bf5f0-3fbc-4d96-95b3-bb951b7f49c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:32.214027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bf5f0-3fbc-4d96-95b3-bb951b7f49c4 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.214318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bf5f0-3fbc-4d96-95b3-bb951b7f49c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.214599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bf5f0-3fbc-4d96-95b3-bb951b7f49c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.215319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bf5f0-3fbc-4d96-95b3-bb951b7f49c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:32.216897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bf5f0-3fbc-4d96-95b3-bb951b7f49c4 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:32.226069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:32.229391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 991/1983] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:56:32.233138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bf5f0-3fbc-4d96-95b3-bb951b7f49c4 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:32.236957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5650a6cf89a46e5b088d5b49a1969e6'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:32.243322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bf5f0-3fbc-4d96-95b3-bb951b7f49c4 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:32.247953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8e065-cff8-4d55-8b57-2ef28eb2e30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:32.248647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8e065-cff8-4d55-8b57-2ef28eb2e30b admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.248932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8e065-cff8-4d55-8b57-2ef28eb2e30b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.249225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8e065-cff8-4d55-8b57-2ef28eb2e30b admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.250032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8e065-cff8-4d55-8b57-2ef28eb2e30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:32.251623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8e065-cff8-4d55-8b57-2ef28eb2e30b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:32.263137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 992/1984] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:56:32.267319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8e065-cff8-4d55-8b57-2ef28eb2e30b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:32.277009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8e065-cff8-4d55-8b57-2ef28eb2e30b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:32.282100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953d561-8acd-43f3-b262-0561b19f33fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:32.282975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953d561-8acd-43f3-b262-0561b19f33fd admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.283257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953d561-8acd-43f3-b262-0561b19f33fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.283531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953d561-8acd-43f3-b262-0561b19f33fd admin admin] PATH_INFO: `/v3/projects/7cb551d8ba024d3d957faed4aa64a063/users/ca390f2d1ee04b699ee649f091a1a900/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.298097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953d561-8acd-43f3-b262-0561b19f33fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb551d8ba024d3d957faed4aa64a063, user_id=ca390f2d1ee04b699ee649f091a1a900, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:32.301127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953d561-8acd-43f3-b262-0561b19f33fd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:32.306237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5650a6cf89a46e5b088d5b49a1969e6'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:32.306909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5650a6cf89a46e5b088d5b49a1969e6'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:32.314553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5650a6cf89a46e5b088d5b49a1969e6'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:32.315210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-165d9345-dec5-4dc2-9519-362f1c8452ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5650a6cf89a46e5b088d5b49a1969e6'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:32.332793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 993/1985] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:31 2022] DELETE /identity/v3/users/f5650a6cf89a46e5b088d5b49a1969e6 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:32.422683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 993/1986] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:32 2022] PUT /identity/v3/projects/7cb551d8ba024d3d957faed4aa64a063/users/ca390f2d1ee04b699ee649f091a1a900/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:32.433146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e677bd4-0079-4f7c-96e7-ae6581a381ce None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.433440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e677bd4-0079-4f7c-96e7-ae6581a381ce None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.433741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e677bd4-0079-4f7c-96e7-ae6581a381ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.461382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-4e677bd4-0079-4f7c-96e7-ae6581a381ce None None] MFA Rules not processed for user `ca390f2d1ee04b699ee649f091a1a900`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:32.462660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4e677bd4-0079-4f7c-96e7-ae6581a381ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:32.474713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:32.528665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 994/1987] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:32.533501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953d561-8acd-43f3-b262-0561b19f33fd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:32.535342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5953d561-8acd-43f3-b262-0561b19f33fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:32.602749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953d561-8acd-43f3-b262-0561b19f33fd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:32.607828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28eba33-965a-470a-a0b5-f4119c00cd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:32.608510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28eba33-965a-470a-a0b5-f4119c00cd0b admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.609178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28eba33-965a-470a-a0b5-f4119c00cd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.609178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28eba33-965a-470a-a0b5-f4119c00cd0b admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.609779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28eba33-965a-470a-a0b5-f4119c00cd0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:32.611711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28eba33-965a-470a-a0b5-f4119c00cd0b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:32.704848 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 994/1988] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:32 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:32.709240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e677bd4-0079-4f7c-96e7-ae6581a381ce None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:32.720077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e677bd4-0079-4f7c-96e7-ae6581a381ce None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:32.782899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d70cd-8cc3-4372-8401-f88363efd278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:32.783620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d70cd-8cc3-4372-8401-f88363efd278 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.783874 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d70cd-8cc3-4372-8401-f88363efd278 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.784154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d70cd-8cc3-4372-8401-f88363efd278 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.784861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d70cd-8cc3-4372-8401-f88363efd278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:32.786910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d70cd-8cc3-4372-8401-f88363efd278 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:32.843201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28eba33-965a-470a-a0b5-f4119c00cd0b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:32.845148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a28eba33-965a-470a-a0b5-f4119c00cd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:32.860306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28eba33-965a-470a-a0b5-f4119c00cd0b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:32.866527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:32.872195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.872195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.872195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] PATH_INFO: `/v3/projects/1c5b55f3ee3d45c8a020595b52c18eff` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.872195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5b55f3ee3d45c8a020595b52c18eff)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:32.872195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:32.908403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 995/1989] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:32 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:32.912351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d70cd-8cc3-4372-8401-f88363efd278 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:32.922053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d70cd-8cc3-4372-8401-f88363efd278 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:32.926819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753e235-1c32-4d26-bece-14b0f0d4fcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:32.927963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753e235-1c32-4d26-bece-14b0f0d4fcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.928250 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753e235-1c32-4d26-bece-14b0f0d4fcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.928523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753e235-1c32-4d26-bece-14b0f0d4fcd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.929277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753e235-1c32-4d26-bece-14b0f0d4fcd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:32.930895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753e235-1c32-4d26-bece-14b0f0d4fcd0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:32.940821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 996/1990] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:56:32.945402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753e235-1c32-4d26-bece-14b0f0d4fcd0 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:32.954511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753e235-1c32-4d26-bece-14b0f0d4fcd0 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:32.959297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d8dd7-21d4-499b-a286-b0e61ee465c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:32.960128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d8dd7-21d4-499b-a286-b0e61ee465c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:32.960410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d8dd7-21d4-499b-a286-b0e61ee465c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:32.960688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d8dd7-21d4-499b-a286-b0e61ee465c0 admin admin] PATH_INFO: `/v3/projects/cd7814e910a0425cba0b6ba40373150c/users/a3a9380b647646c889bf561780230612/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:32.975216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d8dd7-21d4-499b-a286-b0e61ee465c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7814e910a0425cba0b6ba40373150c, user_id=a3a9380b647646c889bf561780230612, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:32.977863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d8dd7-21d4-499b-a286-b0e61ee465c0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.089676 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 997/1991] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:32 2022] PUT /identity/v3/projects/cd7814e910a0425cba0b6ba40373150c/users/a3a9380b647646c889bf561780230612/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:33.093416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d8dd7-21d4-499b-a286-b0e61ee465c0 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:33.098851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5b55f3ee3d45c8a020595b52c18eff', 'request_id': 'req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:33.105172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5b55f3ee3d45c8a020595b52c18eff', 'request_id': 'req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:33.156305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d8dd7-21d4-499b-a286-b0e61ee465c0 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:33.174710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5b55f3ee3d45c8a020595b52c18eff', 'request_id': 'req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:33.177297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:33.188127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] The token cache is being invalidate because project 1c5b55f3ee3d45c8a020595b52c18eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:33.188416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:33.189061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e4138de5-e4cd-4c09-9a7f-6b75c72ce6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:33.190772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 995/1992] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:32 2022] DELETE /identity/v3/projects/1c5b55f3ee3d45c8a020595b52c18eff => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:33.195326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143c547-1838-4a2a-8d7c-01874063622f None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.196221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143c547-1838-4a2a-8d7c-01874063622f None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.196221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143c547-1838-4a2a-8d7c-01874063622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.213172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-9143c547-1838-4a2a-8d7c-01874063622f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:33.214339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9143c547-1838-4a2a-8d7c-01874063622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:33.216709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518c387-c858-4eb8-867a-295407cf658b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:33.217396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518c387-c858-4eb8-867a-295407cf658b admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.217683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518c387-c858-4eb8-867a-295407cf658b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.218045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518c387-c858-4eb8-867a-295407cf658b admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.218782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518c387-c858-4eb8-867a-295407cf658b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:33.220420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518c387-c858-4eb8-867a-295407cf658b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.222711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 996/1993] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:33.231517 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 998/1994] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:56:33.232423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a0f8-8def-4877-8d97-29867a322b78 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.232714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a0f8-8def-4877-8d97-29867a322b78 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.233018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a0f8-8def-4877-8d97-29867a322b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.235485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518c387-c858-4eb8-867a-295407cf658b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:33.237187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c518c387-c858-4eb8-867a-295407cf658b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:33.250658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518c387-c858-4eb8-867a-295407cf658b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:33.251183 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8161a0f8-8def-4877-8d97-29867a322b78 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:33.252439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8161a0f8-8def-4877-8d97-29867a322b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:33.255930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676db60-0e72-484c-8fc1-f36bef2d7047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:33.256830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676db60-0e72-484c-8fc1-f36bef2d7047 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.257118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676db60-0e72-484c-8fc1-f36bef2d7047 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.257391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676db60-0e72-484c-8fc1-f36bef2d7047 admin admin] PATH_INFO: `/v3/projects/cd7814e910a0425cba0b6ba40373150c/users/a3a9380b647646c889bf561780230612/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.259428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676db60-0e72-484c-8fc1-f36bef2d7047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7814e910a0425cba0b6ba40373150c, user_id=a3a9380b647646c889bf561780230612, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:33.260827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 997/1995] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:33.263186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676db60-0e72-484c-8fc1-f36bef2d7047 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.266006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161a0f8-8def-4877-8d97-29867a322b78 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:33.273234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 999/1996] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:33 2022] PUT /identity/v3/projects/cd7814e910a0425cba0b6ba40373150c/users/a3a9380b647646c889bf561780230612/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:33.275075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161a0f8-8def-4877-8d97-29867a322b78 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:33.283717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0011c6b-7ca2-4d9a-be18-3ffe5f070a7c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.284139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0011c6b-7ca2-4d9a-be18-3ffe5f070a7c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.284435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0011c6b-7ca2-4d9a-be18-3ffe5f070a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.315748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-b0011c6b-7ca2-4d9a-be18-3ffe5f070a7c None None] MFA Rules not processed for user `a3a9380b647646c889bf561780230612`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:33.316995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b0011c6b-7ca2-4d9a-be18-3ffe5f070a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:33.336730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33683fa-3fa8-46f1-9801-2604d9501bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:33.337423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33683fa-3fa8-46f1-9801-2604d9501bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.337705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33683fa-3fa8-46f1-9801-2604d9501bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.338064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33683fa-3fa8-46f1-9801-2604d9501bf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.338976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33683fa-3fa8-46f1-9801-2604d9501bf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:56:33.343559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33683fa-3fa8-46f1-9801-2604d9501bf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:33.343559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33683fa-3fa8-46f1-9801-2604d9501bf3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.356301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 998/1997] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:56:33.357699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33683fa-3fa8-46f1-9801-2604d9501bf3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:33.369897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33683fa-3fa8-46f1-9801-2604d9501bf3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:33.374637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8de5d-ed66-4abc-9b6c-0b962e9f395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:33.375293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8de5d-ed66-4abc-9b6c-0b962e9f395d admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.379579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8de5d-ed66-4abc-9b6c-0b962e9f395d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.379579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8de5d-ed66-4abc-9b6c-0b962e9f395d admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.379579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8de5d-ed66-4abc-9b6c-0b962e9f395d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:33.379579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8de5d-ed66-4abc-9b6c-0b962e9f395d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.390160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1000/1998] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:33.401594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0011c6b-7ca2-4d9a-be18-3ffe5f070a7c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:33.403056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b0011c6b-7ca2-4d9a-be18-3ffe5f070a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:33.458141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 999/1999] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:33 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:33.460237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0011c6b-7ca2-4d9a-be18-3ffe5f070a7c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:33.462628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8de5d-ed66-4abc-9b6c-0b962e9f395d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:33.463396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:33.511496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1282028-6076-419b-908b-5d00cda5a9a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:33.512191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1282028-6076-419b-908b-5d00cda5a9a1 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.512477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1282028-6076-419b-908b-5d00cda5a9a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.512757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1282028-6076-419b-908b-5d00cda5a9a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.514669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e1282028-6076-419b-908b-5d00cda5a9a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:33.524597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8de5d-ed66-4abc-9b6c-0b962e9f395d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:33.529913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760e478-d4b3-4439-89b8-0dd9e4c1aed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:33.530620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760e478-d4b3-4439-89b8-0dd9e4c1aed8 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.530903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760e478-d4b3-4439-89b8-0dd9e4c1aed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.531181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760e478-d4b3-4439-89b8-0dd9e4c1aed8 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.531871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760e478-d4b3-4439-89b8-0dd9e4c1aed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:33.533793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760e478-d4b3-4439-89b8-0dd9e4c1aed8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.575549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1282028-6076-419b-908b-5d00cda5a9a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:33.577001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1282028-6076-419b-908b-5d00cda5a9a1 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.586540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1001/2000] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:33.696842 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1000/2001] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:33 2022] POST /identity/v3/users => generated 291 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:33.700955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1282028-6076-419b-908b-5d00cda5a9a1 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:33.712405 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1282028-6076-419b-908b-5d00cda5a9a1 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:33.717054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60a6ed-bb04-48fa-9878-a72e339e6434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:33.717659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60a6ed-bb04-48fa-9878-a72e339e6434 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.717976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60a6ed-bb04-48fa-9878-a72e339e6434 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.718252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60a6ed-bb04-48fa-9878-a72e339e6434 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.718973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60a6ed-bb04-48fa-9878-a72e339e6434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:33.720565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60a6ed-bb04-48fa-9878-a72e339e6434 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.731066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1002/2002] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:33.735078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760e478-d4b3-4439-89b8-0dd9e4c1aed8 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:33.745180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760e478-d4b3-4439-89b8-0dd9e4c1aed8 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:33.749922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fb500-06a5-4c74-bb68-67d74c2b8716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:33.754341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fb500-06a5-4c74-bb68-67d74c2b8716 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.754341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fb500-06a5-4c74-bb68-67d74c2b8716 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.754341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fb500-06a5-4c74-bb68-67d74c2b8716 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.754341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fb500-06a5-4c74-bb68-67d74c2b8716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:33.754341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265fb500-06a5-4c74-bb68-67d74c2b8716 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.763510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1001/2003] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:33.767583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60a6ed-bb04-48fa-9878-a72e339e6434 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:33.778192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60a6ed-bb04-48fa-9878-a72e339e6434 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:33.783171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7658830-b478-462f-be32-eeff978b3fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:33.784144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7658830-b478-462f-be32-eeff978b3fed admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.800769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7658830-b478-462f-be32-eeff978b3fed admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.800769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7658830-b478-462f-be32-eeff978b3fed admin admin] PATH_INFO: `/v3/projects/480724e32b9b4ba388492065a6e106e4/users/57c63bf0a62d4091ad18d390cca159df/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:33.800769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7658830-b478-462f-be32-eeff978b3fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=480724e32b9b4ba388492065a6e106e4, user_id=57c63bf0a62d4091ad18d390cca159df, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:33.803435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7658830-b478-462f-be32-eeff978b3fed admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:33.971601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1003/2004] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:33 2022] PUT /identity/v3/projects/480724e32b9b4ba388492065a6e106e4/users/57c63bf0a62d4091ad18d390cca159df/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:33.981916 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc4684-50b8-465c-a979-6477238cafba None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:33.982370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc4684-50b8-465c-a979-6477238cafba None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:33.982509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc4684-50b8-465c-a979-6477238cafba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:34.009165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0ebc4684-50b8-465c-a979-6477238cafba None None] MFA Rules not processed for user `57c63bf0a62d4091ad18d390cca159df`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:34.018865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0ebc4684-50b8-465c-a979-6477238cafba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:34.076195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1002/2005] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:34.080720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7658830-b478-462f-be32-eeff978b3fed admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:34.137040 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:34.143829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7658830-b478-462f-be32-eeff978b3fed admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:34.148689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4dbdaa-a947-48d6-bc93-991eeda3aaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:34.149360 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4dbdaa-a947-48d6-bc93-991eeda3aaae admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:34.149648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4dbdaa-a947-48d6-bc93-991eeda3aaae admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:34.149969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4dbdaa-a947-48d6-bc93-991eeda3aaae admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:34.150673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4dbdaa-a947-48d6-bc93-991eeda3aaae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:34.152651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4dbdaa-a947-48d6-bc93-991eeda3aaae admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:34.246790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:34.253890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1004/2006] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:34 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:34.258046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc4684-50b8-465c-a979-6477238cafba None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:34.268730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc4684-50b8-465c-a979-6477238cafba None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:34.326223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93c7e4-677e-4732-8bd7-c62bbb5f3a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:34.327052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c7e4-677e-4732-8bd7-c62bbb5f3a8f admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:34.333613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c7e4-677e-4732-8bd7-c62bbb5f3a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:34.333613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c7e4-677e-4732-8bd7-c62bbb5f3a8f admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:34.333613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93c7e4-677e-4732-8bd7-c62bbb5f3a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:34.333613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93c7e4-677e-4732-8bd7-c62bbb5f3a8f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:34.439675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1003/2007] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:34 2022] POST /identity/v3/users => generated 296 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:34.445687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4dbdaa-a947-48d6-bc93-991eeda3aaae admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:34.456623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4dbdaa-a947-48d6-bc93-991eeda3aaae admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:34.461270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab70b6-3ee8-4d26-9284-5159573bc02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:34.464557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab70b6-3ee8-4d26-9284-5159573bc02c admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:34.464557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab70b6-3ee8-4d26-9284-5159573bc02c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:34.464557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab70b6-3ee8-4d26-9284-5159573bc02c admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:34.464557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab70b6-3ee8-4d26-9284-5159573bc02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:34.464961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab70b6-3ee8-4d26-9284-5159573bc02c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:34.479037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1005/2008] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:34.480345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93c7e4-677e-4732-8bd7-c62bbb5f3a8f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:34.492533 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93c7e4-677e-4732-8bd7-c62bbb5f3a8f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:34.497379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8e61e-fd9e-4384-a580-9f51fe94b23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:34.498278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8e61e-fd9e-4384-a580-9f51fe94b23e admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:34.498568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8e61e-fd9e-4384-a580-9f51fe94b23e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:34.498845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8e61e-fd9e-4384-a580-9f51fe94b23e admin admin] PATH_INFO: `/v3/projects/b4bd823111a14921a0843ca906ffe536/users/df4cfe1ba0d44f2d972395c521eecbdb/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:34.514460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8e61e-fd9e-4384-a580-9f51fe94b23e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4bd823111a14921a0843ca906ffe536, user_id=df4cfe1ba0d44f2d972395c521eecbdb, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:34.517200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8e61e-fd9e-4384-a580-9f51fe94b23e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:34.617179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1004/2009] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:34 2022] PUT /identity/v3/projects/b4bd823111a14921a0843ca906ffe536/users/df4cfe1ba0d44f2d972395c521eecbdb/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:34.621180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab70b6-3ee8-4d26-9284-5159573bc02c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:34.631553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab70b6-3ee8-4d26-9284-5159573bc02c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:34.686559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11deb61-457a-4c2c-86fc-18c79a8792b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:34.687229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11deb61-457a-4c2c-86fc-18c79a8792b4 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:34.687516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11deb61-457a-4c2c-86fc-18c79a8792b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:34.687793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11deb61-457a-4c2c-86fc-18c79a8792b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:34.688516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11deb61-457a-4c2c-86fc-18c79a8792b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:34.690142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11deb61-457a-4c2c-86fc-18c79a8792b4 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:34.700273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1006/2010] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:34.704536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8e61e-fd9e-4384-a580-9f51fe94b23e admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:34.715460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8e61e-fd9e-4384-a580-9f51fe94b23e admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:34.720411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733096e3-434d-42fe-86eb-230e4fd5ff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:34.721299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733096e3-434d-42fe-86eb-230e4fd5ff30 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:34.721581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733096e3-434d-42fe-86eb-230e4fd5ff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:34.721895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733096e3-434d-42fe-86eb-230e4fd5ff30 admin admin] PATH_INFO: `/v3/projects/b4bd823111a14921a0843ca906ffe536/users/df4cfe1ba0d44f2d972395c521eecbdb/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:34.723876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733096e3-434d-42fe-86eb-230e4fd5ff30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4bd823111a14921a0843ca906ffe536, user_id=df4cfe1ba0d44f2d972395c521eecbdb, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:34.726575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733096e3-434d-42fe-86eb-230e4fd5ff30 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:34.736674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1005/2011] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:34 2022] PUT /identity/v3/projects/b4bd823111a14921a0843ca906ffe536/users/df4cfe1ba0d44f2d972395c521eecbdb/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:34.747852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b95f5-b21d-40cb-ad6e-bc7f8b2c42b1 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:34.748087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b95f5-b21d-40cb-ad6e-bc7f8b2c42b1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:34.748369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b95f5-b21d-40cb-ad6e-bc7f8b2c42b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:34.775345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-9c7b95f5-b21d-40cb-ad6e-bc7f8b2c42b1 None None] MFA Rules not processed for user `df4cfe1ba0d44f2d972395c521eecbdb`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:34.776558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9c7b95f5-b21d-40cb-ad6e-bc7f8b2c42b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:34.847828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1007/2012] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:34.859086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733096e3-434d-42fe-86eb-230e4fd5ff30 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:34.860838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-733096e3-434d-42fe-86eb-230e4fd5ff30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:34.897209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:34.919075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733096e3-434d-42fe-86eb-230e4fd5ff30 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:34.922758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262ef28-ea84-411d-a4b2-5fdfd74477d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:34.923412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262ef28-ea84-411d-a4b2-5fdfd74477d1 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:34.923701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262ef28-ea84-411d-a4b2-5fdfd74477d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:34.923979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262ef28-ea84-411d-a4b2-5fdfd74477d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:34.936134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262ef28-ea84-411d-a4b2-5fdfd74477d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:34.937671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262ef28-ea84-411d-a4b2-5fdfd74477d1 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:34.948516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1006/2013] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:35.172157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b95f5-b21d-40cb-ad6e-bc7f8b2c42b1 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:35.182702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b95f5-b21d-40cb-ad6e-bc7f8b2c42b1 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:35.186531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5340c1e-ddc0-40bc-9706-0b7781a1a0d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:35.187276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5340c1e-ddc0-40bc-9706-0b7781a1a0d7 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:35.187564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5340c1e-ddc0-40bc-9706-0b7781a1a0d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:35.187845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5340c1e-ddc0-40bc-9706-0b7781a1a0d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:35.247460 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5340c1e-ddc0-40bc-9706-0b7781a1a0d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:35.249035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5340c1e-ddc0-40bc-9706-0b7781a1a0d7 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:35.258640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1008/2014] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:35.791822 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262ef28-ea84-411d-a4b2-5fdfd74477d1 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:35.800525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262ef28-ea84-411d-a4b2-5fdfd74477d1 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:35.804405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:35.805268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:35.805554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:35.805860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:35.807701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:35.821485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:35.866398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:35.867868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:35.876718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1007/2015] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:37.970011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5340c1e-ddc0-40bc-9706-0b7781a1a0d7 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:37.979149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5340c1e-ddc0-40bc-9706-0b7781a1a0d7 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:37.982845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f18b8-a2bb-4192-99dc-798292b0da93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:37.983501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f18b8-a2bb-4192-99dc-798292b0da93 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:37.983788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f18b8-a2bb-4192-99dc-798292b0da93 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:37.984069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f18b8-a2bb-4192-99dc-798292b0da93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:38.036443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:38.050207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f18b8-a2bb-4192-99dc-798292b0da93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:38.051941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f18b8-a2bb-4192-99dc-798292b0da93 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:38.062132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1009/2016] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:45.217234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:45.218725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:45.277284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f18b8-a2bb-4192-99dc-798292b0da93 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:45.286883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156aaec-d126-4527-834d-74105eb4e62e service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:45.292045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f18b8-a2bb-4192-99dc-798292b0da93 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:45.292164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:45.292164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:45.292443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:45.292718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] PATH_INFO: `/v3/users/59d0d1585d3a4959a7434385a15989d7` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:45.293868 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:45.293987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d0d1585d3a4959a7434385a15989d7)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:45.294823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:45.295161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:45.295444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] PATH_INFO: `/v3/users/2ce6c880e4274e97ac1f24bd92c48fc9` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:45.295861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:45.296673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce6c880e4274e97ac1f24bd92c48fc9)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:45.298612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:45.515451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d0d1585d3a4959a7434385a15989d7'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.550410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce6c880e4274e97ac1f24bd92c48fc9'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.620995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d0d1585d3a4959a7434385a15989d7'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.621745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d0d1585d3a4959a7434385a15989d7'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.630028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d0d1585d3a4959a7434385a15989d7'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.631370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-86f2a003-be39-46f3-a712-ce6b07b61bb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d0d1585d3a4959a7434385a15989d7'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.650839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1008/2017] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:45 2022] DELETE /identity/v3/users/59d0d1585d3a4959a7434385a15989d7 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:45.655539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571eb70-97d6-479d-b939-cd29bc2769b4 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:45.655837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571eb70-97d6-479d-b939-cd29bc2769b4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:45.656114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571eb70-97d6-479d-b939-cd29bc2769b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:45.674051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0571eb70-97d6-479d-b939-cd29bc2769b4 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:45.675306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0571eb70-97d6-479d-b939-cd29bc2769b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:45.691135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce6c880e4274e97ac1f24bd92c48fc9'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.692055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce6c880e4274e97ac1f24bd92c48fc9'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.703809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce6c880e4274e97ac1f24bd92c48fc9'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.704556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1d8e3d9f-565a-43e6-a124-71948d6e617f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce6c880e4274e97ac1f24bd92c48fc9'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:45.709932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:45.732832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1010/2018] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:45 2022] DELETE /identity/v3/users/2ce6c880e4274e97ac1f24bd92c48fc9 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:45.736936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477152b-90b6-4d90-bb0d-2b2e65cdae24 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:45.737224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477152b-90b6-4d90-bb0d-2b2e65cdae24 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:45.737502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477152b-90b6-4d90-bb0d-2b2e65cdae24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:45.742350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1009/2019] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:45.752742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571eb70-97d6-479d-b939-cd29bc2769b4 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:45.754675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0571eb70-97d6-479d-b939-cd29bc2769b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:45.757278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d477152b-90b6-4d90-bb0d-2b2e65cdae24 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:45.758471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d477152b-90b6-4d90-bb0d-2b2e65cdae24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:45.828733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571eb70-97d6-479d-b939-cd29bc2769b4 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:45.828923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1011/2020] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:45.840054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477152b-90b6-4d90-bb0d-2b2e65cdae24 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:45.872162 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d07d68-f197-439e-aed1-8a2c9d73c001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:45.872876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d07d68-f197-439e-aed1-8a2c9d73c001 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:45.873156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d07d68-f197-439e-aed1-8a2c9d73c001 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:45.873431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d07d68-f197-439e-aed1-8a2c9d73c001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:45.883897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d07d68-f197-439e-aed1-8a2c9d73c001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:45.885336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d07d68-f197-439e-aed1-8a2c9d73c001 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:45.895214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1010/2021] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:45.897198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477152b-90b6-4d90-bb0d-2b2e65cdae24 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:45.901018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcfb41-4e27-4b5b-b54d-1938b7185bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:45.907003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcfb41-4e27-4b5b-b54d-1938b7185bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:45.907003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcfb41-4e27-4b5b-b54d-1938b7185bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:45.907003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcfb41-4e27-4b5b-b54d-1938b7185bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:45.912755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcfb41-4e27-4b5b-b54d-1938b7185bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:45.914249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcfb41-4e27-4b5b-b54d-1938b7185bd7 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:45.924852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1012/2022] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:46.538829 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d07d68-f197-439e-aed1-8a2c9d73c001 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:46.540480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e0d07d68-f197-439e-aed1-8a2c9d73c001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:46.542828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcfb41-4e27-4b5b-b54d-1938b7185bd7 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:46.544308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-99fcfb41-4e27-4b5b-b54d-1938b7185bd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:46.553002 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:46.555008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d07d68-f197-439e-aed1-8a2c9d73c001 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:46.558767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcfb41-4e27-4b5b-b54d-1938b7185bd7 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:46.560892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:46.566225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:46.566225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:46.566225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] PATH_INFO: `/v3/projects/c00b6dc54f564882b891b36f078b1c07` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:46.566225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c00b6dc54f564882b891b36f078b1c07)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:46.566810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:46.566906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:46.570051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:46.570344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:46.570745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] PATH_INFO: `/v3/projects/28ea3dc3799b4c3591d516d6c65cb935` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:46.572323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=28ea3dc3799b4c3591d516d6c65cb935)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:46.574380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:46.721200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28ea3dc3799b4c3591d516d6c65cb935', 'request_id': 'req-a7ef7fc8-3195-4407-a259-309200300c6e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.721614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c00b6dc54f564882b891b36f078b1c07', 'request_id': 'req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.727311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28ea3dc3799b4c3591d516d6c65cb935', 'request_id': 'req-a7ef7fc8-3195-4407-a259-309200300c6e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.727594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c00b6dc54f564882b891b36f078b1c07', 'request_id': 'req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.799109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28ea3dc3799b4c3591d516d6c65cb935', 'request_id': 'req-a7ef7fc8-3195-4407-a259-309200300c6e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.810881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] The token cache is being invalidate because project 28ea3dc3799b4c3591d516d6c65cb935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:46.811162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.811779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.813502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1013/2023] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:46 2022] DELETE /identity/v3/projects/28ea3dc3799b4c3591d516d6c65cb935 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:46.817283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:46.819161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:46.883691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef7fc8-3195-4407-a259-309200300c6e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:46.913332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c00b6dc54f564882b891b36f078b1c07', 'request_id': 'req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.926603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] The token cache is being invalidate because project c00b6dc54f564882b891b36f078b1c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:46.926905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.928032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-f6acad92-ec37-4b5a-8e64-8c75e2f21d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:46.935153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1011/2024] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:46 2022] DELETE /identity/v3/projects/c00b6dc54f564882b891b36f078b1c07 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:46.939958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a0d47-1a21-4702-a2e2-d2bdf80d7326 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:46.940256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a0d47-1a21-4702-a2e2-d2bdf80d7326 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:46.940544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a0d47-1a21-4702-a2e2-d2bdf80d7326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:46.945607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:46.946507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:46.946792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:46.947066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] PATH_INFO: `/v3/users/ca390f2d1ee04b699ee649f091a1a900` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:46.948464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca390f2d1ee04b699ee649f091a1a900)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:46.950448 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:46.964425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-088a0d47-1a21-4702-a2e2-d2bdf80d7326 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:46.965861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-088a0d47-1a21-4702-a2e2-d2bdf80d7326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:46.980328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1012/2025] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:46.981959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35133f98-e57d-4e8a-ab4c-bff50dfd323b None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:46.982250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35133f98-e57d-4e8a-ab4c-bff50dfd323b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:46.982555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35133f98-e57d-4e8a-ab4c-bff50dfd323b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.001231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-35133f98-e57d-4e8a-ab4c-bff50dfd323b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:47.002536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-35133f98-e57d-4e8a-ab4c-bff50dfd323b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:47.011147 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1013/2026] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:47.022226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35133f98-e57d-4e8a-ab4c-bff50dfd323b None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:47.028235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:47.032295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35133f98-e57d-4e8a-ab4c-bff50dfd323b None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:47.039166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b3091-4b22-40be-bc81-5f7fef772555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:47.040142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3091-4b22-40be-bc81-5f7fef772555 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:47.040620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3091-4b22-40be-bc81-5f7fef772555 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:47.041038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b3091-4b22-40be-bc81-5f7fef772555 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.042200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b3091-4b22-40be-bc81-5f7fef772555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:56:47.042677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b3091-4b22-40be-bc81-5f7fef772555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:47.045564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b3091-4b22-40be-bc81-5f7fef772555 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:47.060410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1014/2027] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:56:47.064617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b3091-4b22-40be-bc81-5f7fef772555 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:47.074609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b3091-4b22-40be-bc81-5f7fef772555 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:47.079220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9986c9-65fa-4347-99b6-b492011de5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:47.079847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9986c9-65fa-4347-99b6-b492011de5de admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:47.080132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9986c9-65fa-4347-99b6-b492011de5de admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:47.080410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9986c9-65fa-4347-99b6-b492011de5de admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.081089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9986c9-65fa-4347-99b6-b492011de5de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:47.083061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9986c9-65fa-4347-99b6-b492011de5de admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:47.147428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:47.156556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca390f2d1ee04b699ee649f091a1a900'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:47.171718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1015/2028] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:47 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:47.175487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9986c9-65fa-4347-99b6-b492011de5de admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:47.239943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9986c9-65fa-4347-99b6-b492011de5de admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:47.244727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f0e3d-d779-4871-83f4-9c96ed0cd0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:47.245397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f0e3d-d779-4871-83f4-9c96ed0cd0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:47.245689 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f0e3d-d779-4871-83f4-9c96ed0cd0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:47.245998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f0e3d-d779-4871-83f4-9c96ed0cd0f0 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.246683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f0e3d-d779-4871-83f4-9c96ed0cd0f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:47.248592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f0e3d-d779-4871-83f4-9c96ed0cd0f0 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:47.256294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca390f2d1ee04b699ee649f091a1a900'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:47.256916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca390f2d1ee04b699ee649f091a1a900'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:47.264525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca390f2d1ee04b699ee649f091a1a900'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:47.265164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca390f2d1ee04b699ee649f091a1a900'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:47.282548 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1014/2029] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:46 2022] DELETE /identity/v3/users/ca390f2d1ee04b699ee649f091a1a900 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:47.376551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1016/2030] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:47 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:47.381098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:47.382521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:47.450699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1d154-32e8-4170-aa00-51e63a62af52 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:47.455238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf090b-7a87-4344-8d46-6f4f826c804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:47.455922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf090b-7a87-4344-8d46-6f4f826c804e admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:47.456417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf090b-7a87-4344-8d46-6f4f826c804e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:47.456574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf090b-7a87-4344-8d46-6f4f826c804e admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.457278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf090b-7a87-4344-8d46-6f4f826c804e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:47.459552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf090b-7a87-4344-8d46-6f4f826c804e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:47.469661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1015/2031] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:47.473930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f0e3d-d779-4871-83f4-9c96ed0cd0f0 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:47.484269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f0e3d-d779-4871-83f4-9c96ed0cd0f0 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:47.488954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db250e5-3268-48ba-ac34-44c354df0c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:47.489664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db250e5-3268-48ba-ac34-44c354df0c1f admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:47.489975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db250e5-3268-48ba-ac34-44c354df0c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:47.490256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db250e5-3268-48ba-ac34-44c354df0c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.490989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db250e5-3268-48ba-ac34-44c354df0c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:47.492727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db250e5-3268-48ba-ac34-44c354df0c1f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:47.503482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1017/2032] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:47.508455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf090b-7a87-4344-8d46-6f4f826c804e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:47.518689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf090b-7a87-4344-8d46-6f4f826c804e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:47.530188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92ec3e-fdea-41a2-882a-59ea25a796cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:47.531116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92ec3e-fdea-41a2-882a-59ea25a796cb admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:47.546293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92ec3e-fdea-41a2-882a-59ea25a796cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:47.546293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92ec3e-fdea-41a2-882a-59ea25a796cb admin admin] PATH_INFO: `/v3/projects/e053fb4f53d741bba2eac12d4d3e386c/users/3358e430bd5a4dda99599d8b42bbd068/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.550348 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92ec3e-fdea-41a2-882a-59ea25a796cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e053fb4f53d741bba2eac12d4d3e386c, user_id=3358e430bd5a4dda99599d8b42bbd068, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:47.555752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92ec3e-fdea-41a2-882a-59ea25a796cb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:47.685923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1016/2033] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:47 2022] PUT /identity/v3/projects/e053fb4f53d741bba2eac12d4d3e386c/users/3358e430bd5a4dda99599d8b42bbd068/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:47.699632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92ec3e-fdea-41a2-882a-59ea25a796cb admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:47.699901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20053f89-ce0c-40ae-9f23-7bf95ebb50d6 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:47.700359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20053f89-ce0c-40ae-9f23-7bf95ebb50d6 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:47.700818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20053f89-ce0c-40ae-9f23-7bf95ebb50d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.701513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2f92ec3e-fdea-41a2-882a-59ea25a796cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:47.727931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-20053f89-ce0c-40ae-9f23-7bf95ebb50d6 None None] MFA Rules not processed for user `3358e430bd5a4dda99599d8b42bbd068`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:47.729228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-20053f89-ce0c-40ae-9f23-7bf95ebb50d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:47.768955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92ec3e-fdea-41a2-882a-59ea25a796cb admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:47.774014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:47.775260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:47.775546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:47.775825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] PATH_INFO: `/v3/users/57c63bf0a62d4091ad18d390cca159df` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.777058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c63bf0a62d4091ad18d390cca159df)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:47.781981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:47.801890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1018/2034] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:47.801890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20053f89-ce0c-40ae-9f23-7bf95ebb50d6 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:47.812177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-20053f89-ce0c-40ae-9f23-7bf95ebb50d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:47.826045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20053f89-ce0c-40ae-9f23-7bf95ebb50d6 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:47.830876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:47.831767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:47.832050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:47.832327 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] PATH_INFO: `/v3/projects/7cb551d8ba024d3d957faed4aa64a063` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:47.833579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb551d8ba024d3d957faed4aa64a063)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:47.835674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:47.926700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:48.028782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:48.050552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb551d8ba024d3d957faed4aa64a063', 'request_id': 'req-1636c583-9ac9-45cd-b93f-162dfe570684'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.051097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c63bf0a62d4091ad18d390cca159df'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.057017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb551d8ba024d3d957faed4aa64a063', 'request_id': 'req-1636c583-9ac9-45cd-b93f-162dfe570684'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.148004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c63bf0a62d4091ad18d390cca159df'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.148647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c63bf0a62d4091ad18d390cca159df'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.156423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c63bf0a62d4091ad18d390cca159df'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.157102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c63bf0a62d4091ad18d390cca159df'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.181967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1017/2035] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:47 2022] DELETE /identity/v3/users/57c63bf0a62d4091ad18d390cca159df => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:48.181967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:48.182260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:48.210901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb551d8ba024d3d957faed4aa64a063', 'request_id': 'req-1636c583-9ac9-45cd-b93f-162dfe570684'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.222473 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] The token cache is being invalidate because project 7cb551d8ba024d3d957faed4aa64a063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:48.222763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.223414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1636c583-9ac9-45cd-b93f-162dfe570684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.225702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1019/2036] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:47 2022] DELETE /identity/v3/projects/7cb551d8ba024d3d957faed4aa64a063 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:48.227024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714f47e-5938-4673-8edd-f5a905bc33ea None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:48.227313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714f47e-5938-4673-8edd-f5a905bc33ea None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:48.227590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714f47e-5938-4673-8edd-f5a905bc33ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:48.239128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953edd5-82fb-4b18-84c7-b11d23abcfea admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:48.243291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:48.243944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:48.244228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:48.244508 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:48.245374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b714f47e-5938-4673-8edd-f5a905bc33ea None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:48.246397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:48.246675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b714f47e-5938-4673-8edd-f5a905bc33ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:48.305691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:48.306485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1020/2037] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:48.307305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:48.307410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714f47e-5938-4673-8edd-f5a905bc33ea None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:48.308662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b714f47e-5938-4673-8edd-f5a905bc33ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:48.316929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1018/2038] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:48.317837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:48.320032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:48.325589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714f47e-5938-4673-8edd-f5a905bc33ea None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:48.327626 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:48.328581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:48.328899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:48.329178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] PATH_INFO: `/v3/users/a3a9380b647646c889bf561780230612` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:48.330422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3a9380b647646c889bf561780230612)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:48.332356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:48.380146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76931c-9d3b-42c5-a1f0-dbe2288cff36 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:48.383993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b42dc-908e-4e1d-aee0-7c198b5e5dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:48.384725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b42dc-908e-4e1d-aee0-7c198b5e5dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:48.384986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b42dc-908e-4e1d-aee0-7c198b5e5dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:48.391722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b42dc-908e-4e1d-aee0-7c198b5e5dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:48.395847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b42dc-908e-4e1d-aee0-7c198b5e5dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:48.397284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b42dc-908e-4e1d-aee0-7c198b5e5dc9 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:48.402375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:48.407413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1019/2039] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:48.600829 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3a9380b647646c889bf561780230612'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.707991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3a9380b647646c889bf561780230612'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.708714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3a9380b647646c889bf561780230612'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.726427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3a9380b647646c889bf561780230612'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.727492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3a9380b647646c889bf561780230612'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:48.755833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1021/2040] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:48 2022] DELETE /identity/v3/users/a3a9380b647646c889bf561780230612 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:48.949146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b42dc-908e-4e1d-aee0-7c198b5e5dc9 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:48.950641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9c6b42dc-908e-4e1d-aee0-7c198b5e5dc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:48.964426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b42dc-908e-4e1d-aee0-7c198b5e5dc9 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:49.032647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:49.033518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:49.033804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:49.034128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] PATH_INFO: `/v3/projects/480724e32b9b4ba388492065a6e106e4` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:49.035917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=480724e32b9b4ba388492065a6e106e4)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:49.038685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:49.161327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480724e32b9b4ba388492065a6e106e4', 'request_id': 'req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.168119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480724e32b9b4ba388492065a6e106e4', 'request_id': 'req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.273421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480724e32b9b4ba388492065a6e106e4', 'request_id': 'req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.285870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] The token cache is being invalidate because project 480724e32b9b4ba388492065a6e106e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:49.286228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.287511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.289167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1020/2041] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:48 2022] DELETE /identity/v3/projects/480724e32b9b4ba388492065a6e106e4 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:49.293364 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:49.294791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:49.308035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d3546-7aba-4544-b32b-bb5646f47e39 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:49.340232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:49.366697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:49.368580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:49.368993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:49.369296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] PATH_INFO: `/v3/users/df4cfe1ba0d44f2d972395c521eecbdb` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:49.370639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4cfe1ba0d44f2d972395c521eecbdb)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:49.375201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:49.375631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:49.384392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:49.398178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23a2c0-9afe-43c9-b505-cdc506ff4e70 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:49.403766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:49.404643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:49.404998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:49.405193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] PATH_INFO: `/v3/projects/cd7814e910a0425cba0b6ba40373150c` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:49.406413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7814e910a0425cba0b6ba40373150c)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:49.411206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:49.588027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7814e910a0425cba0b6ba40373150c', 'request_id': 'req-67294851-6320-4827-a3d5-b0c47b306c47'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.588717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4cfe1ba0d44f2d972395c521eecbdb'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.593363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7814e910a0425cba0b6ba40373150c', 'request_id': 'req-67294851-6320-4827-a3d5-b0c47b306c47'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.665594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4cfe1ba0d44f2d972395c521eecbdb'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.666270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4cfe1ba0d44f2d972395c521eecbdb'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.673589 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4cfe1ba0d44f2d972395c521eecbdb'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.674331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c69a7865-a016-4701-926d-65f8502b9b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4cfe1ba0d44f2d972395c521eecbdb'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.693122 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1022/2042] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:56:49 2022] DELETE /identity/v3/users/df4cfe1ba0d44f2d972395c521eecbdb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:49.730976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7814e910a0425cba0b6ba40373150c', 'request_id': 'req-67294851-6320-4827-a3d5-b0c47b306c47'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.743564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] The token cache is being invalidate because project cd7814e910a0425cba0b6ba40373150c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:49.743864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.744581 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-67294851-6320-4827-a3d5-b0c47b306c47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:49.746345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1021/2043] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:49 2022] DELETE /identity/v3/projects/cd7814e910a0425cba0b6ba40373150c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:49.751272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41612216-ca6e-4ee1-a1b8-1d50427b6e47 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:49.751765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41612216-ca6e-4ee1-a1b8-1d50427b6e47 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:49.751880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41612216-ca6e-4ee1-a1b8-1d50427b6e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:49.770011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-41612216-ca6e-4ee1-a1b8-1d50427b6e47 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:49.771316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-41612216-ca6e-4ee1-a1b8-1d50427b6e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:49.831354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1023/2044] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:49.839286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729c22e-3e18-4f04-99ea-12a39a1ee449 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:49.839520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729c22e-3e18-4f04-99ea-12a39a1ee449 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:49.839800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729c22e-3e18-4f04-99ea-12a39a1ee449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:49.858699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-9729c22e-3e18-4f04-99ea-12a39a1ee449 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:49.859903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9729c22e-3e18-4f04-99ea-12a39a1ee449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:49.868320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1022/2045] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:49.872519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41612216-ca6e-4ee1-a1b8-1d50427b6e47 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:49.882097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41612216-ca6e-4ee1-a1b8-1d50427b6e47 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:49.886720 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149de6a-cb2b-4d1a-94dd-081349aa3c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:49.887399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149de6a-cb2b-4d1a-94dd-081349aa3c7c admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:49.887682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149de6a-cb2b-4d1a-94dd-081349aa3c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:49.887955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149de6a-cb2b-4d1a-94dd-081349aa3c7c admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:49.888860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149de6a-cb2b-4d1a-94dd-081349aa3c7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:56:49.889172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149de6a-cb2b-4d1a-94dd-081349aa3c7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:49.890882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149de6a-cb2b-4d1a-94dd-081349aa3c7c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:49.902943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1024/2046] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:56:49.907275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729c22e-3e18-4f04-99ea-12a39a1ee449 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:49.918427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729c22e-3e18-4f04-99ea-12a39a1ee449 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:49.924546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf7a53-8821-4b2e-9030-f2d2a735cdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:49.925393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf7a53-8821-4b2e-9030-f2d2a735cdf7 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:49.925577 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf7a53-8821-4b2e-9030-f2d2a735cdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:49.926117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf7a53-8821-4b2e-9030-f2d2a735cdf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:49.926860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf7a53-8821-4b2e-9030-f2d2a735cdf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:49.929648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf7a53-8821-4b2e-9030-f2d2a735cdf7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:49.933339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:50.061770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1023/2047] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:49 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:50.065887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149de6a-cb2b-4d1a-94dd-081349aa3c7c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:50.076149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149de6a-cb2b-4d1a-94dd-081349aa3c7c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:50.114320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:50.132372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65fd6-e617-46c6-90b6-38c0dd59a646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:50.133011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65fd6-e617-46c6-90b6-38c0dd59a646 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.133425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65fd6-e617-46c6-90b6-38c0dd59a646 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.133712 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc65fd6-e617-46c6-90b6-38c0dd59a646 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.134449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc65fd6-e617-46c6-90b6-38c0dd59a646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:50.136378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc65fd6-e617-46c6-90b6-38c0dd59a646 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:50.210355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf7a53-8821-4b2e-9030-f2d2a735cdf7 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:50.212216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-25cf7a53-8821-4b2e-9030-f2d2a735cdf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:50.233543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf7a53-8821-4b2e-9030-f2d2a735cdf7 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:50.233543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:50.233543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.233543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.239267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1025/2048] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:50 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:50.244371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] PATH_INFO: `/v3/projects/b4bd823111a14921a0843ca906ffe536` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.246815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4bd823111a14921a0843ca906ffe536)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:50.246940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65fd6-e617-46c6-90b6-38c0dd59a646 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:50.248951 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:50.260070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc65fd6-e617-46c6-90b6-38c0dd59a646 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:50.269876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100b01-81fc-4081-840f-dc8c8ab28cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:50.269876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100b01-81fc-4081-840f-dc8c8ab28cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.269876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100b01-81fc-4081-840f-dc8c8ab28cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.269876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100b01-81fc-4081-840f-dc8c8ab28cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.270438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d100b01-81fc-4081-840f-dc8c8ab28cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:50.270438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d100b01-81fc-4081-840f-dc8c8ab28cb3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:50.281857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1026/2049] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:50.284362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100b01-81fc-4081-840f-dc8c8ab28cb3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:50.294272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100b01-81fc-4081-840f-dc8c8ab28cb3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:50.299130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbffa93-2cce-47a2-8f9a-01e8674f03dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:50.301547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbffa93-2cce-47a2-8f9a-01e8674f03dd admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.301547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbffa93-2cce-47a2-8f9a-01e8674f03dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.301547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbffa93-2cce-47a2-8f9a-01e8674f03dd admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.301547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbffa93-2cce-47a2-8f9a-01e8674f03dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:50.303465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbffa93-2cce-47a2-8f9a-01e8674f03dd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:50.313609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1027/2050] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:56:50.317715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbffa93-2cce-47a2-8f9a-01e8674f03dd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:50.327237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbffa93-2cce-47a2-8f9a-01e8674f03dd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:50.332704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455339bf-bb0d-485f-b560-44090413b99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:50.334132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455339bf-bb0d-485f-b560-44090413b99c admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.334418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455339bf-bb0d-485f-b560-44090413b99c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.334762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455339bf-bb0d-485f-b560-44090413b99c admin admin] PATH_INFO: `/v3/projects/8edf005aaa274e98a3f098e45105c0da/users/a84d1283721f408d85c0b835d9173daf/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.350254 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455339bf-bb0d-485f-b560-44090413b99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edf005aaa274e98a3f098e45105c0da, user_id=a84d1283721f408d85c0b835d9173daf, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:50.354632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455339bf-bb0d-485f-b560-44090413b99c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:50.367628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4bd823111a14921a0843ca906ffe536', 'request_id': 'req-1b61b161-378b-441d-bd6d-6e84150901b8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:50.373362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4bd823111a14921a0843ca906ffe536', 'request_id': 'req-1b61b161-378b-441d-bd6d-6e84150901b8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:50.531380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4bd823111a14921a0843ca906ffe536', 'request_id': 'req-1b61b161-378b-441d-bd6d-6e84150901b8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:50.538995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1028/2051] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:50 2022] PUT /identity/v3/projects/8edf005aaa274e98a3f098e45105c0da/users/a84d1283721f408d85c0b835d9173daf/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:50.543983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] The token cache is being invalidate because project b4bd823111a14921a0843ca906ffe536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:56:50.544175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05f2ff-8b6d-4ee4-844c-b65d660e1d63 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.544305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:50.544638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05f2ff-8b6d-4ee4-844c-b65d660e1d63 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.544916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-1b61b161-378b-441d-bd6d-6e84150901b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:56:50.545019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05f2ff-8b6d-4ee4-844c-b65d660e1d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.546580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1024/2052] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:56:50 2022] DELETE /identity/v3/projects/b4bd823111a14921a0843ca906ffe536 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:50.554026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3370ddb-7dd6-441d-8120-ef6a2cd56399 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.554344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3370ddb-7dd6-441d-8120-ef6a2cd56399 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.554621 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3370ddb-7dd6-441d-8120-ef6a2cd56399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.564960 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-4e05f2ff-8b6d-4ee4-844c-b65d660e1d63 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:50.566413 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4e05f2ff-8b6d-4ee4-844c-b65d660e1d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:50.574052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a3370ddb-7dd6-441d-8120-ef6a2cd56399 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:50.575257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a3370ddb-7dd6-441d-8120-ef6a2cd56399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:50.634910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1029/2053] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:50.637899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1025/2054] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:56:50.645469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3370ddb-7dd6-441d-8120-ef6a2cd56399 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:50.645637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b3e4f-5e78-481c-9c4c-d8a482caf5e4 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.646529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b3e4f-5e78-481c-9c4c-d8a482caf5e4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.646810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b3e4f-5e78-481c-9c4c-d8a482caf5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.647588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a3370ddb-7dd6-441d-8120-ef6a2cd56399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:50.664866 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-401b3e4f-5e78-481c-9c4c-d8a482caf5e4 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:50.666093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-401b3e4f-5e78-481c-9c4c-d8a482caf5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:50.674120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1030/2055] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:50.678701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b3e4f-5e78-481c-9c4c-d8a482caf5e4 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:50.687490 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b3e4f-5e78-481c-9c4c-d8a482caf5e4 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:50.692304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccf21a-9ca6-4650-bf39-fdc6acde5955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:50.693487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccf21a-9ca6-4650-bf39-fdc6acde5955 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.693762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccf21a-9ca6-4650-bf39-fdc6acde5955 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.694173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ccf21a-9ca6-4650-bf39-fdc6acde5955 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.694906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccf21a-9ca6-4650-bf39-fdc6acde5955 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:56:50.695345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccf21a-9ca6-4650-bf39-fdc6acde5955 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:50.696879 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ccf21a-9ca6-4650-bf39-fdc6acde5955 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:50.735976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1031/2056] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:56:50.736205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3370ddb-7dd6-441d-8120-ef6a2cd56399 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:50.736205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b422b-68c8-420f-81c7-2338ff1bd248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:50.736205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b422b-68c8-420f-81c7-2338ff1bd248 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.736205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b422b-68c8-420f-81c7-2338ff1bd248 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.736583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b422b-68c8-420f-81c7-2338ff1bd248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.736665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccf21a-9ca6-4650-bf39-fdc6acde5955 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:50.736745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b422b-68c8-420f-81c7-2338ff1bd248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:50.736745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b422b-68c8-420f-81c7-2338ff1bd248 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:50.736899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ccf21a-9ca6-4650-bf39-fdc6acde5955 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:50.737318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1026/2057] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:50.742160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715d2a9-1d30-4090-939f-dafd7b6e6f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:50.742845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715d2a9-1d30-4090-939f-dafd7b6e6f19 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:50.743121 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715d2a9-1d30-4090-939f-dafd7b6e6f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:50.743459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715d2a9-1d30-4090-939f-dafd7b6e6f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:50.744086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715d2a9-1d30-4090-939f-dafd7b6e6f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:50.746064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715d2a9-1d30-4090-939f-dafd7b6e6f19 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:51.024326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1032/2058] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:50 2022] POST /identity/v3/projects => generated 377 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:51.032568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b422b-68c8-420f-81c7-2338ff1bd248 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:51.046563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b422b-68c8-420f-81c7-2338ff1bd248 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:51.055419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:51.108498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2b054-6583-4c79-abe3-84789344637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:51.109186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2b054-6583-4c79-abe3-84789344637d admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:51.109467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2b054-6583-4c79-abe3-84789344637d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:51.109739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2b054-6583-4c79-abe3-84789344637d admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:51.110480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2b054-6583-4c79-abe3-84789344637d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:51.112435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2b054-6583-4c79-abe3-84789344637d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:51.219166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1027/2059] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:51 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:51.220445 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715d2a9-1d30-4090-939f-dafd7b6e6f19 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:51.232427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715d2a9-1d30-4090-939f-dafd7b6e6f19 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:51.237958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e65f5-941f-4bdb-af65-56b9dbcb3ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:51.238648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e65f5-941f-4bdb-af65-56b9dbcb3ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:51.238935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e65f5-941f-4bdb-af65-56b9dbcb3ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:51.239261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e65f5-941f-4bdb-af65-56b9dbcb3ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:51.240820 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e65f5-941f-4bdb-af65-56b9dbcb3ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:51.242498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e65f5-941f-4bdb-af65-56b9dbcb3ef9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:51.255858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1033/2060] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:51.260457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2b054-6583-4c79-abe3-84789344637d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:51.275446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2b054-6583-4c79-abe3-84789344637d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:51.284964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39153e-8012-4414-adce-d0fd656276c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:51.284964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39153e-8012-4414-adce-d0fd656276c6 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:51.284964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39153e-8012-4414-adce-d0fd656276c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:51.284964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39153e-8012-4414-adce-d0fd656276c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:51.285353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39153e-8012-4414-adce-d0fd656276c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:51.286010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39153e-8012-4414-adce-d0fd656276c6 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:51.299734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1028/2061] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:51.304242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e65f5-941f-4bdb-af65-56b9dbcb3ef9 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:51.315252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e65f5-941f-4bdb-af65-56b9dbcb3ef9 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:51.320857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df00657-4a3a-43e0-b604-443f35191411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:51.326418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df00657-4a3a-43e0-b604-443f35191411 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:51.326418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df00657-4a3a-43e0-b604-443f35191411 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:51.326418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df00657-4a3a-43e0-b604-443f35191411 admin admin] PATH_INFO: `/v3/projects/b036252bdcb4474997e7af0511acd65d/users/d1ceac900f8f4a0983b24df28f60fbe1/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:51.340727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df00657-4a3a-43e0-b604-443f35191411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b036252bdcb4474997e7af0511acd65d, user_id=d1ceac900f8f4a0983b24df28f60fbe1, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:51.343412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df00657-4a3a-43e0-b604-443f35191411 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:51.461717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1034/2062] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:51 2022] PUT /identity/v3/projects/b036252bdcb4474997e7af0511acd65d/users/d1ceac900f8f4a0983b24df28f60fbe1/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:56:51.472535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29f3b7-ba12-4d56-b927-c2df44d80c77 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:51.472756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29f3b7-ba12-4d56-b927-c2df44d80c77 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:51.473039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29f3b7-ba12-4d56-b927-c2df44d80c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:51.501062 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ff29f3b7-ba12-4d56-b927-c2df44d80c77 None None] MFA Rules not processed for user `d1ceac900f8f4a0983b24df28f60fbe1`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:51.502322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ff29f3b7-ba12-4d56-b927-c2df44d80c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:51.592431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1029/2063] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:51.593557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df00657-4a3a-43e0-b604-443f35191411 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:51.672746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df00657-4a3a-43e0-b604-443f35191411 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:51.679256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea4e12-b064-466d-bb37-b4df32708b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:51.679540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea4e12-b064-466d-bb37-b4df32708b5b admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:51.679633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea4e12-b064-466d-bb37-b4df32708b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:51.679896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea4e12-b064-466d-bb37-b4df32708b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:51.680623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea4e12-b064-466d-bb37-b4df32708b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:51.682602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea4e12-b064-466d-bb37-b4df32708b5b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:51.775829 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:51.783417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1035/2064] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:56:51 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:56:51.787455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29f3b7-ba12-4d56-b927-c2df44d80c77 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:51.798651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29f3b7-ba12-4d56-b927-c2df44d80c77 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:51.860847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61656252-1dc8-4e6d-9647-51263718eb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:51.861527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61656252-1dc8-4e6d-9647-51263718eb37 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:51.862214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61656252-1dc8-4e6d-9647-51263718eb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:51.862501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61656252-1dc8-4e6d-9647-51263718eb37 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:51.863215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61656252-1dc8-4e6d-9647-51263718eb37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:51.866061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61656252-1dc8-4e6d-9647-51263718eb37 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:52.014954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1030/2065] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:56:51 2022] POST /identity/v3/users => generated 295 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:56:52.018818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea4e12-b064-466d-bb37-b4df32708b5b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:52.029249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea4e12-b064-466d-bb37-b4df32708b5b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:52.034036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a5470-448f-4dd2-8e7a-b704a4f48b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:52.037175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a5470-448f-4dd2-8e7a-b704a4f48b47 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:52.037175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a5470-448f-4dd2-8e7a-b704a4f48b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:52.037175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a5470-448f-4dd2-8e7a-b704a4f48b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:52.037175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a5470-448f-4dd2-8e7a-b704a4f48b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:52.038252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a5470-448f-4dd2-8e7a-b704a4f48b47 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:52.054281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1036/2066] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:52.055124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61656252-1dc8-4e6d-9647-51263718eb37 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:52.065839 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61656252-1dc8-4e6d-9647-51263718eb37 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:52.071083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492673cc-188b-4a3f-a413-485241a12424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:52.072069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492673cc-188b-4a3f-a413-485241a12424 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:52.072424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492673cc-188b-4a3f-a413-485241a12424 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:52.072775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492673cc-188b-4a3f-a413-485241a12424 admin admin] PATH_INFO: `/v3/projects/29c9aa20451b4a3583ae775897111862/users/0ef5869a993245fbbf05947d013de6c1/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:52.089350 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492673cc-188b-4a3f-a413-485241a12424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c9aa20451b4a3583ae775897111862, user_id=0ef5869a993245fbbf05947d013de6c1, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:52.092053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492673cc-188b-4a3f-a413-485241a12424 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:52.163617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1031/2067] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:52 2022] PUT /identity/v3/projects/29c9aa20451b4a3583ae775897111862/users/0ef5869a993245fbbf05947d013de6c1/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:52.167579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a5470-448f-4dd2-8e7a-b704a4f48b47 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:52.178312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a5470-448f-4dd2-8e7a-b704a4f48b47 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:52.236954 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c25e0-7dd9-478b-822f-a4d01928f3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:52.237644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c25e0-7dd9-478b-822f-a4d01928f3ec admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:52.237978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c25e0-7dd9-478b-822f-a4d01928f3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:52.238689 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c25e0-7dd9-478b-822f-a4d01928f3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:52.238962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c25e0-7dd9-478b-822f-a4d01928f3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:52.241173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c25e0-7dd9-478b-822f-a4d01928f3ec admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:52.251409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1037/2068] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:56:52.255300 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492673cc-188b-4a3f-a413-485241a12424 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:52.267397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492673cc-188b-4a3f-a413-485241a12424 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:52.278426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa069e27-9c27-449e-a682-2aa5be5b12ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:52.278426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa069e27-9c27-449e-a682-2aa5be5b12ba admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:52.278426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa069e27-9c27-449e-a682-2aa5be5b12ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:52.278426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa069e27-9c27-449e-a682-2aa5be5b12ba admin admin] PATH_INFO: `/v3/projects/29c9aa20451b4a3583ae775897111862/users/0ef5869a993245fbbf05947d013de6c1/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:52.278748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa069e27-9c27-449e-a682-2aa5be5b12ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c9aa20451b4a3583ae775897111862, user_id=0ef5869a993245fbbf05947d013de6c1, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:52.278748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa069e27-9c27-449e-a682-2aa5be5b12ba admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:52.290854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1032/2069] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:56:52 2022] PUT /identity/v3/projects/29c9aa20451b4a3583ae775897111862/users/0ef5869a993245fbbf05947d013de6c1/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:56:52.302501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e436fb-a724-4876-a3c8-825cea48958c None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:52.304621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e436fb-a724-4876-a3c8-825cea48958c None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:52.304974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e436fb-a724-4876-a3c8-825cea48958c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:52.335969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c7e436fb-a724-4876-a3c8-825cea48958c None None] MFA Rules not processed for user `0ef5869a993245fbbf05947d013de6c1`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:52.337400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c7e436fb-a724-4876-a3c8-825cea48958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:52.405724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:52.408175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47883ca3-fb94-4f9b-9264-1ae212df3554 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:52.408541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47883ca3-fb94-4f9b-9264-1ae212df3554 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:52.408953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47883ca3-fb94-4f9b-9264-1ae212df3554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:52.426732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1038/2070] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:52.438554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e436fb-a724-4876-a3c8-825cea48958c None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:52.439965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c7e436fb-a724-4876-a3c8-825cea48958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:52.450402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-47883ca3-fb94-4f9b-9264-1ae212df3554 None None] MFA Rules not processed for user `a84d1283721f408d85c0b835d9173daf`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:52.451738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-47883ca3-fb94-4f9b-9264-1ae212df3554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:52.475552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:56:52.520538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e436fb-a724-4876-a3c8-825cea48958c None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:52.525444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2b77a-2e6d-4fdc-ab58-fce80450e821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:52.526360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b77a-2e6d-4fdc-ab58-fce80450e821 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:52.526653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b77a-2e6d-4fdc-ab58-fce80450e821 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:52.526940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b77a-2e6d-4fdc-ab58-fce80450e821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:52.538026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1033/2071] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:52.586859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2b77a-2e6d-4fdc-ab58-fce80450e821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:52.588316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2b77a-2e6d-4fdc-ab58-fce80450e821 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:52.597494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1039/2072] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 15:56:52.875627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47883ca3-fb94-4f9b-9264-1ae212df3554 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:52.877083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-47883ca3-fb94-4f9b-9264-1ae212df3554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:52.888928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47883ca3-fb94-4f9b-9264-1ae212df3554 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:52.922000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242014d9-4d9e-405c-8701-c0943c766b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:52.922000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242014d9-4d9e-405c-8701-c0943c766b7b service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:52.922000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242014d9-4d9e-405c-8701-c0943c766b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:52.922000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242014d9-4d9e-405c-8701-c0943c766b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:52.922901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242014d9-4d9e-405c-8701-c0943c766b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:52.922901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242014d9-4d9e-405c-8701-c0943c766b7b service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:52.922901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1034/2073] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:56:54.919471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663d11e-89c3-4ef7-8d10-c07a70f749c8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:54.919937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663d11e-89c3-4ef7-8d10-c07a70f749c8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:54.920045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663d11e-89c3-4ef7-8d10-c07a70f749c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:54.938275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a663d11e-89c3-4ef7-8d10-c07a70f749c8 None None] MFA Rules not processed for user `a84d1283721f408d85c0b835d9173daf`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:56:54.939493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a663d11e-89c3-4ef7-8d10-c07a70f749c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:56:54.951020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1040/2074] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:56:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:56:59.492996 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242014d9-4d9e-405c-8701-c0943c766b7b service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:56:59.506617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242014d9-4d9e-405c-8701-c0943c766b7b service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:56:59.506617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1140ef-5ff5-4d45-ac3f-574f872cc8c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:56:59.506617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1140ef-5ff5-4d45-ac3f-574f872cc8c2 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:56:59.506617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1140ef-5ff5-4d45-ac3f-574f872cc8c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:56:59.506953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1140ef-5ff5-4d45-ac3f-574f872cc8c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:56:59.515911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1140ef-5ff5-4d45-ac3f-574f872cc8c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:56:59.517347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1140ef-5ff5-4d45-ac3f-574f872cc8c2 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:56:59.527611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1035/2075] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:56:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:04.933222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=50 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:07.251940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663d11e-89c3-4ef7-8d10-c07a70f749c8 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:07.253385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a663d11e-89c3-4ef7-8d10-c07a70f749c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:07.318881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663d11e-89c3-4ef7-8d10-c07a70f749c8 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:07.323688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:07.324570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:07.324854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:07.325134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] PATH_INFO: `/v3/users/3358e430bd5a4dda99599d8b42bbd068` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:07.326423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3358e430bd5a4dda99599d8b42bbd068)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:07.328350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:07.619300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3358e430bd5a4dda99599d8b42bbd068'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:07.708566 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3358e430bd5a4dda99599d8b42bbd068'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:07.709308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3358e430bd5a4dda99599d8b42bbd068'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:07.717088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3358e430bd5a4dda99599d8b42bbd068'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:07.717865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3358e430bd5a4dda99599d8b42bbd068'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:07.737744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1041/2076] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:57:07 2022] DELETE /identity/v3/users/3358e430bd5a4dda99599d8b42bbd068 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:07.742300 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fac13-f7e1-4e7e-b5ab-bd4e907ee866 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:07.742803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fac13-f7e1-4e7e-b5ab-bd4e907ee866 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:07.742900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fac13-f7e1-4e7e-b5ab-bd4e907ee866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:07.762155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-7b9fac13-f7e1-4e7e-b5ab-bd4e907ee866 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:07.763437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7b9fac13-f7e1-4e7e-b5ab-bd4e907ee866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:07.834053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1036/2077] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:07.843945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:07.845461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:07.909306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eda10d-3cae-481a-be4c-04baab1bf0f8 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:07.913622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8a8cb-554f-4825-b1fd-7c6fe0c3c4e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:07.916461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8a8cb-554f-4825-b1fd-7c6fe0c3c4e9 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:07.916461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8a8cb-554f-4825-b1fd-7c6fe0c3c4e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:07.916461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8a8cb-554f-4825-b1fd-7c6fe0c3c4e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:07.924400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8a8cb-554f-4825-b1fd-7c6fe0c3c4e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:07.926178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8a8cb-554f-4825-b1fd-7c6fe0c3c4e9 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:07.936768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1042/2078] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:08.452855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fac13-f7e1-4e7e-b5ab-bd4e907ee866 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:08.454829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7b9fac13-f7e1-4e7e-b5ab-bd4e907ee866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:08.469496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fac13-f7e1-4e7e-b5ab-bd4e907ee866 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:08.474922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:08.475792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:08.476077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:08.476355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] PATH_INFO: `/v3/projects/e053fb4f53d741bba2eac12d4d3e386c` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:08.477704 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e053fb4f53d741bba2eac12d4d3e386c)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:08.479542 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:08.645701 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:08.673672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e053fb4f53d741bba2eac12d4d3e386c', 'request_id': 'req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:08.680674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e053fb4f53d741bba2eac12d4d3e386c', 'request_id': 'req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:08.798824 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e053fb4f53d741bba2eac12d4d3e386c', 'request_id': 'req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:08.812907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] The token cache is being invalidate because project e053fb4f53d741bba2eac12d4d3e386c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:57:08.813514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:08.816103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-42c8db95-1a3b-40f9-8430-92f7a5a3a15a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:08.818694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1037/2079] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:57:08 2022] DELETE /identity/v3/projects/e053fb4f53d741bba2eac12d4d3e386c => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:08.823783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd7bc-716a-43a7-a874-7fc3c16660a0 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:08.824175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd7bc-716a-43a7-a874-7fc3c16660a0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:08.827997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd7bc-716a-43a7-a874-7fc3c16660a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:08.850249 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-3c8dd7bc-716a-43a7-a874-7fc3c16660a0 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:08.851499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3c8dd7bc-716a-43a7-a874-7fc3c16660a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:08.871394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:08.939856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1043/2080] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:08.939946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d176e-6886-4f89-b79d-1d259fb7d774 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:08.939946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d176e-6886-4f89-b79d-1d259fb7d774 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:08.939946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d176e-6886-4f89-b79d-1d259fb7d774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:08.964526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d32d176e-6886-4f89-b79d-1d259fb7d774 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:08.966151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d32d176e-6886-4f89-b79d-1d259fb7d774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:08.974822 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1038/2081] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:57:08.979366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dd7bc-716a-43a7-a874-7fc3c16660a0 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:08.990258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dd7bc-716a-43a7-a874-7fc3c16660a0 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:08.995354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757463ba-c74b-422c-8d10-73d042cad5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:08.996166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757463ba-c74b-422c-8d10-73d042cad5ae admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:08.996414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757463ba-c74b-422c-8d10-73d042cad5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:08.996697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757463ba-c74b-422c-8d10-73d042cad5ae admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:08.997450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757463ba-c74b-422c-8d10-73d042cad5ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:57:08.997918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757463ba-c74b-422c-8d10-73d042cad5ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:08.999383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757463ba-c74b-422c-8d10-73d042cad5ae admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:09.012693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1044/2082] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:57:09.017724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d176e-6886-4f89-b79d-1d259fb7d774 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:09.027057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d176e-6886-4f89-b79d-1d259fb7d774 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:09.031700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119eeb-6083-46eb-b2b3-721365ee43a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:09.032470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119eeb-6083-46eb-b2b3-721365ee43a1 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:09.032758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119eeb-6083-46eb-b2b3-721365ee43a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:09.033053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d119eeb-6083-46eb-b2b3-721365ee43a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:09.033734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d119eeb-6083-46eb-b2b3-721365ee43a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:09.035687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d119eeb-6083-46eb-b2b3-721365ee43a1 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:09.125948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1039/2083] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:57:09 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:09.130271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757463ba-c74b-422c-8d10-73d042cad5ae admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:09.141107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757463ba-c74b-422c-8d10-73d042cad5ae admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:09.196135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c1286-3ab9-4507-85f2-46c8575a0783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:09.196803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c1286-3ab9-4507-85f2-46c8575a0783 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:09.197087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c1286-3ab9-4507-85f2-46c8575a0783 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:09.197362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c1286-3ab9-4507-85f2-46c8575a0783 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:09.198193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c1286-3ab9-4507-85f2-46c8575a0783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:09.200204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c1286-3ab9-4507-85f2-46c8575a0783 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:09.279559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1045/2084] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:57:09 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:09.283714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119eeb-6083-46eb-b2b3-721365ee43a1 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:09.294645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d119eeb-6083-46eb-b2b3-721365ee43a1 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:09.299394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797f996-97df-49a7-a8cd-bd9a01182353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:09.300061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797f996-97df-49a7-a8cd-bd9a01182353 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:09.300357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797f996-97df-49a7-a8cd-bd9a01182353 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:09.300887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797f996-97df-49a7-a8cd-bd9a01182353 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:09.301704 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797f996-97df-49a7-a8cd-bd9a01182353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:09.303994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797f996-97df-49a7-a8cd-bd9a01182353 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:09.314211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1040/2085] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:09.318166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c1286-3ab9-4507-85f2-46c8575a0783 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:09.328056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c1286-3ab9-4507-85f2-46c8575a0783 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:09.332864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c2ae5-ee24-4777-a483-ed6d085e5748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:09.333525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c2ae5-ee24-4777-a483-ed6d085e5748 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:09.333830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c2ae5-ee24-4777-a483-ed6d085e5748 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:09.334131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c2ae5-ee24-4777-a483-ed6d085e5748 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:09.334861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c2ae5-ee24-4777-a483-ed6d085e5748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:09.336461 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c2ae5-ee24-4777-a483-ed6d085e5748 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:09.346311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1046/2086] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:09.350762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797f996-97df-49a7-a8cd-bd9a01182353 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:09.360731 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797f996-97df-49a7-a8cd-bd9a01182353 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:09.366187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8ed6-4827-46bc-bec3-127f755d62ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:09.367080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8ed6-4827-46bc-bec3-127f755d62ca admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:09.369094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8ed6-4827-46bc-bec3-127f755d62ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:09.369094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8ed6-4827-46bc-bec3-127f755d62ca admin admin] PATH_INFO: `/v3/projects/d16ae801ca124180b78f2fce16acc4f7/users/54fcab272d7946c69aac92f6d3030942/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:09.382050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8ed6-4827-46bc-bec3-127f755d62ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16ae801ca124180b78f2fce16acc4f7, user_id=54fcab272d7946c69aac92f6d3030942, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:09.384703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8ed6-4827-46bc-bec3-127f755d62ca admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:09.438652 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1041/2087] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:57:09 2022] PUT /identity/v3/projects/d16ae801ca124180b78f2fce16acc4f7/users/54fcab272d7946c69aac92f6d3030942/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:09.443715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fa636-90f6-42fd-b0e5-e8d605b8f092 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:09.444002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fa636-90f6-42fd-b0e5-e8d605b8f092 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:09.444282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fa636-90f6-42fd-b0e5-e8d605b8f092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:09.461965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-105fa636-90f6-42fd-b0e5-e8d605b8f092 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:09.463186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-105fa636-90f6-42fd-b0e5-e8d605b8f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:09.522098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1047/2088] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:09.531934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8ed6-4827-46bc-bec3-127f755d62ca admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:09.533625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1c0f8ed6-4827-46bc-bec3-127f755d62ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:09.591144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8ed6-4827-46bc-bec3-127f755d62ca admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:09.594858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede984d-bf16-4dc8-a3c9-b56506655c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:09.595507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede984d-bf16-4dc8-a3c9-b56506655c63 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:09.595799 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede984d-bf16-4dc8-a3c9-b56506655c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:09.596076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede984d-bf16-4dc8-a3c9-b56506655c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:09.605745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede984d-bf16-4dc8-a3c9-b56506655c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:09.607220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede984d-bf16-4dc8-a3c9-b56506655c63 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:09.617242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1042/2089] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:11.078705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bbdaf-0c16-4b69-bc37-e2957dafade3 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:11.079198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bbdaf-0c16-4b69-bc37-e2957dafade3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:11.080240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bbdaf-0c16-4b69-bc37-e2957dafade3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:11.113521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cd4bbdaf-0c16-4b69-bc37-e2957dafade3 None None] MFA Rules not processed for user `54fcab272d7946c69aac92f6d3030942`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:11.115244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cd4bbdaf-0c16-4b69-bc37-e2957dafade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:11.146045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:11.208053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1048/2090] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:11.225873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede984d-bf16-4dc8-a3c9-b56506655c63 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:11.227380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1ede984d-bf16-4dc8-a3c9-b56506655c63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:11.290420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede984d-bf16-4dc8-a3c9-b56506655c63 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:11.294080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357888f-0a89-43db-af7e-08eba6c0e4c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:11.294737 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357888f-0a89-43db-af7e-08eba6c0e4c3 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:11.295048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357888f-0a89-43db-af7e-08eba6c0e4c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:11.295323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357888f-0a89-43db-af7e-08eba6c0e4c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:11.306699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357888f-0a89-43db-af7e-08eba6c0e4c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:11.308937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357888f-0a89-43db-af7e-08eba6c0e4c3 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:11.313583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:11.325950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1043/2091] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:13.447760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f9f6f-89ae-4a13-b120-4d7a9ec62230 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:13.448248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f9f6f-89ae-4a13-b120-4d7a9ec62230 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:13.450232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f9f6f-89ae-4a13-b120-4d7a9ec62230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:13.469528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cb9f9f6f-89ae-4a13-b120-4d7a9ec62230 None None] MFA Rules not processed for user `54fcab272d7946c69aac92f6d3030942`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:13.471612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cb9f9f6f-89ae-4a13-b120-4d7a9ec62230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:13.489218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1049/2092] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:16.798708 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444297f-1f97-4200-b998-595796d8b921 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:16.799203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444297f-1f97-4200-b998-595796d8b921 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:16.799299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444297f-1f97-4200-b998-595796d8b921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:16.818238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-3444297f-1f97-4200-b998-595796d8b921 None None] MFA Rules not processed for user `c4fd1155ff11417db8c647f0aadf74a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:16.819451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3444297f-1f97-4200-b998-595796d8b921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:16.924285 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1044/2093] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:23.463825 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=22 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:26.915555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=25 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:28.936806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f9f6f-89ae-4a13-b120-4d7a9ec62230 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:28.938304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cb9f9f6f-89ae-4a13-b120-4d7a9ec62230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:28.951737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f9f6f-89ae-4a13-b120-4d7a9ec62230 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:28.956241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:28.957109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:28.957395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:28.957673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] PATH_INFO: `/v3/users/d1ceac900f8f4a0983b24df28f60fbe1` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:28.958899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ceac900f8f4a0983b24df28f60fbe1)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:28.960891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:29.309977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ceac900f8f4a0983b24df28f60fbe1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:29.385066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ceac900f8f4a0983b24df28f60fbe1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:29.387706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ceac900f8f4a0983b24df28f60fbe1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:29.394742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ceac900f8f4a0983b24df28f60fbe1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:29.396230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ceac900f8f4a0983b24df28f60fbe1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:29.414536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1050/2094] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:57:28 2022] DELETE /identity/v3/users/d1ceac900f8f4a0983b24df28f60fbe1 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:29.418830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc318c-09b5-45fa-9504-96024f56c66b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:29.419319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc318c-09b5-45fa-9504-96024f56c66b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:29.419436 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc318c-09b5-45fa-9504-96024f56c66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:29.438639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f7bc318c-09b5-45fa-9504-96024f56c66b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:29.440106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f7bc318c-09b5-45fa-9504-96024f56c66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:29.517095 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1045/2095] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:29.535654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:29.535654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:29.600710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736fc3-82e8-4881-9a67-1dae655767e3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:29.606218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c852bf-5b1d-4dcb-8eb9-be5933d46454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:29.606897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c852bf-5b1d-4dcb-8eb9-be5933d46454 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:29.607228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c852bf-5b1d-4dcb-8eb9-be5933d46454 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:29.607505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c852bf-5b1d-4dcb-8eb9-be5933d46454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:29.617434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c852bf-5b1d-4dcb-8eb9-be5933d46454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:29.618908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c852bf-5b1d-4dcb-8eb9-be5933d46454 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:29.637320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1051/2096] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:30.508564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc318c-09b5-45fa-9504-96024f56c66b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:30.511010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f7bc318c-09b5-45fa-9504-96024f56c66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:30.528094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc318c-09b5-45fa-9504-96024f56c66b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:30.534550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:30.535485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:30.536347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:30.537510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] PATH_INFO: `/v3/projects/b036252bdcb4474997e7af0511acd65d` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:30.541149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b036252bdcb4474997e7af0511acd65d)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:30.553087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:30.730496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b036252bdcb4474997e7af0511acd65d', 'request_id': 'req-e279da73-7865-44c3-92c9-b36098fefe77'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:30.742043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b036252bdcb4474997e7af0511acd65d', 'request_id': 'req-e279da73-7865-44c3-92c9-b36098fefe77'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:30.832590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b036252bdcb4474997e7af0511acd65d', 'request_id': 'req-e279da73-7865-44c3-92c9-b36098fefe77'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:30.844208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] The token cache is being invalidate because project b036252bdcb4474997e7af0511acd65d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:57:30.844497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:30.845117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e279da73-7865-44c3-92c9-b36098fefe77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:30.846716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1046/2097] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:57:30 2022] DELETE /identity/v3/projects/b036252bdcb4474997e7af0511acd65d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:30.850934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c852bf-5b1d-4dcb-8eb9-be5933d46454 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:30.860564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c852bf-5b1d-4dcb-8eb9-be5933d46454 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:30.919979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:30.920858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:30.921139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:30.921417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] PATH_INFO: `/v3/users/0ef5869a993245fbbf05947d013de6c1` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:30.922655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef5869a993245fbbf05947d013de6c1)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:30.924592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:30.941910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:31.221996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef5869a993245fbbf05947d013de6c1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:31.381261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef5869a993245fbbf05947d013de6c1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:31.381922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef5869a993245fbbf05947d013de6c1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:31.392058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef5869a993245fbbf05947d013de6c1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:31.393462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef5869a993245fbbf05947d013de6c1'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:31.418807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1052/2098] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:57:30 2022] DELETE /identity/v3/users/0ef5869a993245fbbf05947d013de6c1 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:31.807617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ae1eb-35da-4699-85a5-4382beb5df66 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:31.808220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ae1eb-35da-4699-85a5-4382beb5df66 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:31.808220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ae1eb-35da-4699-85a5-4382beb5df66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:31.827025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d10ae1eb-35da-4699-85a5-4382beb5df66 None None] MFA Rules not processed for user `a84d1283721f408d85c0b835d9173daf`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:31.828262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d10ae1eb-35da-4699-85a5-4382beb5df66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:31.894500 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1047/2099] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:32.542405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:32.544494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:32.623787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476c329-2c66-47eb-80c3-f4ac09412954 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:32.634970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:32.635864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:32.639219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:32.639219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] PATH_INFO: `/v3/projects/29c9aa20451b4a3583ae775897111862` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:32.639219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c9aa20451b4a3583ae775897111862)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:32.640432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:32.789013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c9aa20451b4a3583ae775897111862', 'request_id': 'req-ab187793-4a38-481b-a310-c93051731925'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:32.795242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c9aa20451b4a3583ae775897111862', 'request_id': 'req-ab187793-4a38-481b-a310-c93051731925'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:32.886080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c9aa20451b4a3583ae775897111862', 'request_id': 'req-ab187793-4a38-481b-a310-c93051731925'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:32.909765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] The token cache is being invalidate because project 29c9aa20451b4a3583ae775897111862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:57:32.910579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:32.912358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ab187793-4a38-481b-a310-c93051731925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:32.915303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1053/2100] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:57:32 2022] DELETE /identity/v3/projects/29c9aa20451b4a3583ae775897111862 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:32.920349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227b022-7152-4c22-bb74-8fe5760628de None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:32.920867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227b022-7152-4c22-bb74-8fe5760628de None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:32.920980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227b022-7152-4c22-bb74-8fe5760628de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:32.940616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-9227b022-7152-4c22-bb74-8fe5760628de None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:32.942249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9227b022-7152-4c22-bb74-8fe5760628de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:32.997865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:33.073986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1048/2101] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:33.078012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb19dd-b039-462e-aeb5-fb0e73f31b44 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:33.078527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb19dd-b039-462e-aeb5-fb0e73f31b44 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:33.078816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb19dd-b039-462e-aeb5-fb0e73f31b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:33.110870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7fdb19dd-b039-462e-aeb5-fb0e73f31b44 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:33.112606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7fdb19dd-b039-462e-aeb5-fb0e73f31b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:33.127646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1054/2102] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:33.132147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227b022-7152-4c22-bb74-8fe5760628de None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:33.146524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227b022-7152-4c22-bb74-8fe5760628de None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:33.151527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322772c2-bbd4-49ba-9ccc-639259c05e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:33.152134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322772c2-bbd4-49ba-9ccc-639259c05e2a admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:33.152423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322772c2-bbd4-49ba-9ccc-639259c05e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:33.152704 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322772c2-bbd4-49ba-9ccc-639259c05e2a admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:33.153603 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322772c2-bbd4-49ba-9ccc-639259c05e2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:57:33.153946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322772c2-bbd4-49ba-9ccc-639259c05e2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:33.155586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322772c2-bbd4-49ba-9ccc-639259c05e2a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:33.187498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1049/2103] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:57:33.192112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb19dd-b039-462e-aeb5-fb0e73f31b44 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:33.204923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb19dd-b039-462e-aeb5-fb0e73f31b44 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:33.218676 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca0fbc-2980-417d-a933-6f2a49996dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:33.219449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca0fbc-2980-417d-a933-6f2a49996dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:33.219741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca0fbc-2980-417d-a933-6f2a49996dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:33.224814 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca0fbc-2980-417d-a933-6f2a49996dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:33.224814 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca0fbc-2980-417d-a933-6f2a49996dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:33.225341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca0fbc-2980-417d-a933-6f2a49996dd2 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:33.262606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:33.376397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1055/2104] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:57:33 2022] POST /identity/v3/projects => generated 381 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:57:33.380684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322772c2-bbd4-49ba-9ccc-639259c05e2a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:33.390933 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322772c2-bbd4-49ba-9ccc-639259c05e2a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:33.454898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd026b-4b46-498c-a022-18d489b7e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:33.458463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd026b-4b46-498c-a022-18d489b7e925 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:33.458463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd026b-4b46-498c-a022-18d489b7e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:33.458463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd026b-4b46-498c-a022-18d489b7e925 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:33.458463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd026b-4b46-498c-a022-18d489b7e925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:33.458840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd026b-4b46-498c-a022-18d489b7e925 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:33.571937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1050/2105] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:57:33 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:33.576123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca0fbc-2980-417d-a933-6f2a49996dd2 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:33.592840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca0fbc-2980-417d-a933-6f2a49996dd2 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:33.597923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43346d-99de-4c31-8c54-564f93dd9417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:33.599210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43346d-99de-4c31-8c54-564f93dd9417 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:33.599498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43346d-99de-4c31-8c54-564f93dd9417 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:33.599779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43346d-99de-4c31-8c54-564f93dd9417 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:33.600519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43346d-99de-4c31-8c54-564f93dd9417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:33.602331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43346d-99de-4c31-8c54-564f93dd9417 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:33.618454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1056/2106] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:33.618629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd026b-4b46-498c-a022-18d489b7e925 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:33.632183 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd026b-4b46-498c-a022-18d489b7e925 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:33.640693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c634-6420-44b4-b5a5-f0af0d29936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:33.641483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c634-6420-44b4-b5a5-f0af0d29936d admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:33.642159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c634-6420-44b4-b5a5-f0af0d29936d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:33.642532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c634-6420-44b4-b5a5-f0af0d29936d admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:33.643632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c634-6420-44b4-b5a5-f0af0d29936d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:33.647440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c634-6420-44b4-b5a5-f0af0d29936d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:33.664801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1051/2107] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:57:33.668742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43346d-99de-4c31-8c54-564f93dd9417 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:33.680031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43346d-99de-4c31-8c54-564f93dd9417 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:33.697023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f5a8c-115d-4abe-ab3a-d9c1d6252b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:33.698150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f5a8c-115d-4abe-ab3a-d9c1d6252b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:33.698452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f5a8c-115d-4abe-ab3a-d9c1d6252b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:33.698726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f5a8c-115d-4abe-ab3a-d9c1d6252b7c admin admin] PATH_INFO: `/v3/projects/ca3b12d2accb43d0aa95c5562f9eddb9/users/7625ca9538384151babd7ac28dd824dc/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:33.717869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f5a8c-115d-4abe-ab3a-d9c1d6252b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3b12d2accb43d0aa95c5562f9eddb9, user_id=7625ca9538384151babd7ac28dd824dc, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:33.721421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f5a8c-115d-4abe-ab3a-d9c1d6252b7c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:33.815481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1057/2108] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:57:33 2022] PUT /identity/v3/projects/ca3b12d2accb43d0aa95c5562f9eddb9/users/7625ca9538384151babd7ac28dd824dc/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:33.826915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bfec2-4f48-42fa-915e-129a5b5e09a3 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:33.827603 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bfec2-4f48-42fa-915e-129a5b5e09a3 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:33.828127 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bfec2-4f48-42fa-915e-129a5b5e09a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:33.880266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-992bfec2-4f48-42fa-915e-129a5b5e09a3 None None] MFA Rules not processed for user `7625ca9538384151babd7ac28dd824dc`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:33.881858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-992bfec2-4f48-42fa-915e-129a5b5e09a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:33.985497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1052/2109] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:33.990362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f5a8c-115d-4abe-ab3a-d9c1d6252b7c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:34.103802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f5a8c-115d-4abe-ab3a-d9c1d6252b7c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:34.109154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d71233-c3f0-4e8f-a96a-07c4ba9ee9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:34.110094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d71233-c3f0-4e8f-a96a-07c4ba9ee9bf admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:34.110601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d71233-c3f0-4e8f-a96a-07c4ba9ee9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:34.111014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d71233-c3f0-4e8f-a96a-07c4ba9ee9bf admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:34.111863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d71233-c3f0-4e8f-a96a-07c4ba9ee9bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:34.114692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d71233-c3f0-4e8f-a96a-07c4ba9ee9bf admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:34.231220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1058/2110] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:57:33 2022] POST /identity/v3/projects => generated 381 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:34.239180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bfec2-4f48-42fa-915e-129a5b5e09a3 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:34.263627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bfec2-4f48-42fa-915e-129a5b5e09a3 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:34.288908 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:34.333505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe09fd9-a569-45d1-bd16-fd2c6765147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:34.334236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe09fd9-a569-45d1-bd16-fd2c6765147e admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:34.334519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe09fd9-a569-45d1-bd16-fd2c6765147e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:34.334794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe09fd9-a569-45d1-bd16-fd2c6765147e admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:34.335491 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe09fd9-a569-45d1-bd16-fd2c6765147e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:34.337412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe09fd9-a569-45d1-bd16-fd2c6765147e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:34.534617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1053/2111] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:57:34 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:34.538660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d71233-c3f0-4e8f-a96a-07c4ba9ee9bf admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:34.548899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d71233-c3f0-4e8f-a96a-07c4ba9ee9bf admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:34.554615 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237587b5-d8e5-4a77-b409-53e1e30c075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:34.555282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237587b5-d8e5-4a77-b409-53e1e30c075c admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:34.555562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237587b5-d8e5-4a77-b409-53e1e30c075c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:34.555838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237587b5-d8e5-4a77-b409-53e1e30c075c admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:34.556564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237587b5-d8e5-4a77-b409-53e1e30c075c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:34.558239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237587b5-d8e5-4a77-b409-53e1e30c075c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:34.571379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1059/2112] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:34.575754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe09fd9-a569-45d1-bd16-fd2c6765147e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:34.590058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe09fd9-a569-45d1-bd16-fd2c6765147e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:34.596311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b0df5-95b0-4da1-9c5c-78926cc7b8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:34.597242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b0df5-95b0-4da1-9c5c-78926cc7b8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:34.597540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b0df5-95b0-4da1-9c5c-78926cc7b8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:34.597796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b0df5-95b0-4da1-9c5c-78926cc7b8c4 admin admin] PATH_INFO: `/v3/projects/a170d1b0927c421a95c48efd37baa775/users/182431e5a31c4097aa6f6535837297f8/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:34.614283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b0df5-95b0-4da1-9c5c-78926cc7b8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a170d1b0927c421a95c48efd37baa775, user_id=182431e5a31c4097aa6f6535837297f8, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:34.617086 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b0df5-95b0-4da1-9c5c-78926cc7b8c4 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:34.712869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1054/2113] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:57:34 2022] PUT /identity/v3/projects/a170d1b0927c421a95c48efd37baa775/users/182431e5a31c4097aa6f6535837297f8/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:34.717105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237587b5-d8e5-4a77-b409-53e1e30c075c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:34.727930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237587b5-d8e5-4a77-b409-53e1e30c075c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:34.785004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45bfc5-d255-4d7c-8630-ea5f580c4f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:34.786194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45bfc5-d255-4d7c-8630-ea5f580c4f8f admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:34.786480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45bfc5-d255-4d7c-8630-ea5f580c4f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:34.791011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45bfc5-d255-4d7c-8630-ea5f580c4f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:34.791011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45bfc5-d255-4d7c-8630-ea5f580c4f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:34.791011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45bfc5-d255-4d7c-8630-ea5f580c4f8f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:34.799276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1060/2114] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:34.804340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b0df5-95b0-4da1-9c5c-78926cc7b8c4 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:34.814653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b0df5-95b0-4da1-9c5c-78926cc7b8c4 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:34.819284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538de78-aef2-4b5c-9a84-ae0054656af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:34.820170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538de78-aef2-4b5c-9a84-ae0054656af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:34.820452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538de78-aef2-4b5c-9a84-ae0054656af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:34.820735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538de78-aef2-4b5c-9a84-ae0054656af3 admin admin] PATH_INFO: `/v3/projects/a170d1b0927c421a95c48efd37baa775/users/182431e5a31c4097aa6f6535837297f8/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:34.822732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538de78-aef2-4b5c-9a84-ae0054656af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a170d1b0927c421a95c48efd37baa775, user_id=182431e5a31c4097aa6f6535837297f8, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:34.825380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538de78-aef2-4b5c-9a84-ae0054656af3 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:34.835516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1055/2115] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:57:34 2022] PUT /identity/v3/projects/a170d1b0927c421a95c48efd37baa775/users/182431e5a31c4097aa6f6535837297f8/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:34.845924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c8814-4c68-47d2-9a14-6a513b683ba1 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:34.846294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c8814-4c68-47d2-9a14-6a513b683ba1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:34.846636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c8814-4c68-47d2-9a14-6a513b683ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:34.875170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f26c8814-4c68-47d2-9a14-6a513b683ba1 None None] MFA Rules not processed for user `182431e5a31c4097aa6f6535837297f8`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:34.876403 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f26c8814-4c68-47d2-9a14-6a513b683ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:34.894417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:34.944187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1061/2116] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:34.956791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538de78-aef2-4b5c-9a84-ae0054656af3 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:34.958960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c538de78-aef2-4b5c-9a84-ae0054656af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:35.022339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538de78-aef2-4b5c-9a84-ae0054656af3 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:35.026146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50838fb-653a-47ba-9279-3dd2af5c0867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:35.026805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50838fb-653a-47ba-9279-3dd2af5c0867 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:35.027094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50838fb-653a-47ba-9279-3dd2af5c0867 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:35.027372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50838fb-653a-47ba-9279-3dd2af5c0867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:35.037923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50838fb-653a-47ba-9279-3dd2af5c0867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:35.039625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50838fb-653a-47ba-9279-3dd2af5c0867 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:35.049907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1056/2117] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:36.133836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c8814-4c68-47d2-9a14-6a513b683ba1 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:36.142882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c8814-4c68-47d2-9a14-6a513b683ba1 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:36.147688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d8e2b-bdc9-4311-a103-9e283c8b9bdb tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '182431e5a31c4097aa6f6535837297f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a170d1b0927c421a95c48efd37baa775', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:36.148425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d8e2b-bdc9-4311-a103-9e283c8b9bdb tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:36.148719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d8e2b-bdc9-4311-a103-9e283c8b9bdb tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:36.149014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d8e2b-bdc9-4311-a103-9e283c8b9bdb tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:36.149756 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d8e2b-bdc9-4311-a103-9e283c8b9bdb tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:57:36.150104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d8e2b-bdc9-4311-a103-9e283c8b9bdb tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:36.151737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d8e2b-bdc9-4311-a103-9e283c8b9bdb tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:36.164390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1062/2118] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:57:36.173136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50838fb-653a-47ba-9279-3dd2af5c0867 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:36.181231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50838fb-653a-47ba-9279-3dd2af5c0867 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:36.186307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bbe62-586c-4771-94ab-31fcf854e28b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '182431e5a31c4097aa6f6535837297f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a170d1b0927c421a95c48efd37baa775', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:36.187003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bbe62-586c-4771-94ab-31fcf854e28b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:36.187297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bbe62-586c-4771-94ab-31fcf854e28b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:36.187606 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bbe62-586c-4771-94ab-31fcf854e28b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:36.188356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bbe62-586c-4771-94ab-31fcf854e28b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:36.190339 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bbe62-586c-4771-94ab-31fcf854e28b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:36.316122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1057/2119] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:57:36 2022] POST /identity/v3/projects => generated 355 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:36.320442 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d8e2b-bdc9-4311-a103-9e283c8b9bdb tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:36.333376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d8e2b-bdc9-4311-a103-9e283c8b9bdb tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:36.391624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '182431e5a31c4097aa6f6535837297f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a170d1b0927c421a95c48efd37baa775', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:36.392321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:36.392608 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:36.392891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:36.393650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:57:36.393997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:36.395697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:36.406853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1063/2120] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:57:36.448810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bbe62-586c-4771-94ab-31fcf854e28b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:36.460674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bbe62-586c-4771-94ab-31fcf854e28b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:36.467545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '182431e5a31c4097aa6f6535837297f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a170d1b0927c421a95c48efd37baa775', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:36.468429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:36.468726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:36.469017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] PATH_INFO: `/v3/projects/eabe2006bd5a43f78106ac00d2dc8210` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:36.470247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Authorizing `identity:delete_project(project_id=eabe2006bd5a43f78106ac00d2dc8210)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:36.472188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:36.664625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eabe2006bd5a43f78106ac00d2dc8210', 'request_id': 'req-adfe3b9e-d13d-400c-a18a-92f00101b212'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:36.669640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:36.678114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eabe2006bd5a43f78106ac00d2dc8210', 'request_id': 'req-adfe3b9e-d13d-400c-a18a-92f00101b212'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:36.775815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eabe2006bd5a43f78106ac00d2dc8210', 'request_id': 'req-adfe3b9e-d13d-400c-a18a-92f00101b212'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:36.791780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] The token cache is being invalidate because project eabe2006bd5a43f78106ac00d2dc8210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:57:36.792105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:36.792785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-adfe3b9e-d13d-400c-a18a-92f00101b212 tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:36.794513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1058/2121] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:57:36 2022] DELETE /identity/v3/projects/eabe2006bd5a43f78106ac00d2dc8210 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:37.714950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:37.716942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:37.880374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d855b-498f-4051-bf01-8b5e0ab56d0b tempest-VolumeQuotaClassesTest-1239740803 tempest-VolumeQuotaClassesTest-1239740803-project-admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:37.957763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:38.000503 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:38.001686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:38.002246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:38.003164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] PATH_INFO: `/v3/users/7625ca9538384151babd7ac28dd824dc` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:38.008744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7625ca9538384151babd7ac28dd824dc)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:38.013157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:38.131464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e338ae6-0d49-408b-a2dc-14e5ac78d69a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:38.131992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e338ae6-0d49-408b-a2dc-14e5ac78d69a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:38.132085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e338ae6-0d49-408b-a2dc-14e5ac78d69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:38.154365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-1e338ae6-0d49-408b-a2dc-14e5ac78d69a None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:38.156327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1e338ae6-0d49-408b-a2dc-14e5ac78d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:38.239981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1059/2122] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:57:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:57:38.306667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e338ae6-0d49-408b-a2dc-14e5ac78d69a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:38.308120 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1e338ae6-0d49-408b-a2dc-14e5ac78d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:38.341108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7625ca9538384151babd7ac28dd824dc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:38.413594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e338ae6-0d49-408b-a2dc-14e5ac78d69a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:38.415528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7625ca9538384151babd7ac28dd824dc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:38.417476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7625ca9538384151babd7ac28dd824dc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:38.423016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec8336-df9e-4893-9ee8-31bec6c3e78c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:38.424034 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec8336-df9e-4893-9ee8-31bec6c3e78c service swift] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:38.424332 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec8336-df9e-4893-9ee8-31bec6c3e78c service swift] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:38.424616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec8336-df9e-4893-9ee8-31bec6c3e78c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:38.427489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-61ec8336-df9e-4893-9ee8-31bec6c3e78c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:38.435231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7625ca9538384151babd7ac28dd824dc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:38.436041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7625ca9538384151babd7ac28dd824dc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:38.440462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec8336-df9e-4893-9ee8-31bec6c3e78c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:38.443179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec8336-df9e-4893-9ee8-31bec6c3e78c service swift] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:38.452818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1060/2123] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:57:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 15:57:38.466236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1064/2124] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:57:37 2022] DELETE /identity/v3/users/7625ca9538384151babd7ac28dd824dc => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:38.470990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b650716-33f7-4c65-b33a-e10e914d9ece None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:38.471311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b650716-33f7-4c65-b33a-e10e914d9ece None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:38.471504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b650716-33f7-4c65-b33a-e10e914d9ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:38.537082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0b650716-33f7-4c65-b33a-e10e914d9ece None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:38.538410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0b650716-33f7-4c65-b33a-e10e914d9ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:38.639267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1061/2125] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:57:38.659991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:38.662083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:38.822485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328959e-a3c6-4b5f-a300-346571e4e50c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:38.830480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba2644-2a12-40e1-b6c1-bbbb8278fb44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:38.831253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba2644-2a12-40e1-b6c1-bbbb8278fb44 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:38.832084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba2644-2a12-40e1-b6c1-bbbb8278fb44 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:38.833580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba2644-2a12-40e1-b6c1-bbbb8278fb44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:38.850470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba2644-2a12-40e1-b6c1-bbbb8278fb44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:38.852014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba2644-2a12-40e1-b6c1-bbbb8278fb44 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:38.866028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1065/2126] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:39.316585 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97df8be-b8d2-4b40-afff-a007239aec15 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:39.320119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97df8be-b8d2-4b40-afff-a007239aec15 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:39.320119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97df8be-b8d2-4b40-afff-a007239aec15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:39.417230 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-c97df8be-b8d2-4b40-afff-a007239aec15 None None] MFA Rules not processed for user `81ca45714ee44a549fd002b4aa13973b`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:39.454711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c97df8be-b8d2-4b40-afff-a007239aec15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:39.492834 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:39.571859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1062/2127] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:39.849119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba2644-2a12-40e1-b6c1-bbbb8278fb44 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:39.851484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-31ba2644-2a12-40e1-b6c1-bbbb8278fb44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:39.887045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba2644-2a12-40e1-b6c1-bbbb8278fb44 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:39.900912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:39.901997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:39.902483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:39.903009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] PATH_INFO: `/v3/projects/ca3b12d2accb43d0aa95c5562f9eddb9` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:39.912342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca3b12d2accb43d0aa95c5562f9eddb9)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:39.914380 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:40.235906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca3b12d2accb43d0aa95c5562f9eddb9', 'request_id': 'req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:40.261094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca3b12d2accb43d0aa95c5562f9eddb9', 'request_id': 'req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:40.261094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:40.388404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca3b12d2accb43d0aa95c5562f9eddb9', 'request_id': 'req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:40.414354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] The token cache is being invalidate because project ca3b12d2accb43d0aa95c5562f9eddb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:57:40.414649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:40.417375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:40.422528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1066/2128] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:57:39 2022] DELETE /identity/v3/projects/ca3b12d2accb43d0aa95c5562f9eddb9 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:40.429675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97df8be-b8d2-4b40-afff-a007239aec15 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:40.454466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97df8be-b8d2-4b40-afff-a007239aec15 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:40.631780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:40.633856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:40.634685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:40.635209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] PATH_INFO: `/v3/users/182431e5a31c4097aa6f6535837297f8` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:40.639933 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=182431e5a31c4097aa6f6535837297f8)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:40.647896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:40.975926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182431e5a31c4097aa6f6535837297f8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:41.073023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182431e5a31c4097aa6f6535837297f8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:41.073658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182431e5a31c4097aa6f6535837297f8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:41.085244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182431e5a31c4097aa6f6535837297f8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:41.086469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a9d20d42-aec9-4515-8cb4-20c70321c1b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182431e5a31c4097aa6f6535837297f8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:41.141319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1063/2129] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:57:40 2022] DELETE /identity/v3/users/182431e5a31c4097aa6f6535837297f8 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:42.217601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:42.220550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:42.253649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d3d5e-6fc2-4d19-b88c-e46d7d30bfb8 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:42.328933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:42.330380 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:42.330933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:42.331313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] PATH_INFO: `/v3/projects/a170d1b0927c421a95c48efd37baa775` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:42.338564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a170d1b0927c421a95c48efd37baa775)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:42.342084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:42.480039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a170d1b0927c421a95c48efd37baa775', 'request_id': 'req-3c64a080-fa72-425e-bac4-50fe66f4bcfb'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:42.489336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a170d1b0927c421a95c48efd37baa775', 'request_id': 'req-3c64a080-fa72-425e-bac4-50fe66f4bcfb'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:42.568788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a170d1b0927c421a95c48efd37baa775', 'request_id': 'req-3c64a080-fa72-425e-bac4-50fe66f4bcfb'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:42.588277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] The token cache is being invalidate because project a170d1b0927c421a95c48efd37baa775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:57:42.589120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:42.590811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3c64a080-fa72-425e-bac4-50fe66f4bcfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:42.592810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1067/2130] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:57:42 2022] DELETE /identity/v3/projects/a170d1b0927c421a95c48efd37baa775 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:42.597929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648523-5821-42d7-92f0-03aff1756b5b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:42.599363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648523-5821-42d7-92f0-03aff1756b5b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:42.600033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06648523-5821-42d7-92f0-03aff1756b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:42.631759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-06648523-5821-42d7-92f0-03aff1756b5b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:42.635254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-06648523-5821-42d7-92f0-03aff1756b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:42.706119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:42.745878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1064/2131] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:42.751944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414525-b1bb-4470-a383-f71676366379 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:42.761909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414525-b1bb-4470-a383-f71676366379 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:42.761909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed414525-b1bb-4470-a383-f71676366379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:42.796342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ed414525-b1bb-4470-a383-f71676366379 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:42.796524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ed414525-b1bb-4470-a383-f71676366379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:42.807304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1068/2132] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:57:42.811692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648523-5821-42d7-92f0-03aff1756b5b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:42.825718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06648523-5821-42d7-92f0-03aff1756b5b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:42.837204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe19ad2-e254-41b4-88cb-e7336ed02860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:42.838495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe19ad2-e254-41b4-88cb-e7336ed02860 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:42.838863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe19ad2-e254-41b4-88cb-e7336ed02860 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:42.839216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe19ad2-e254-41b4-88cb-e7336ed02860 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:42.840056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe19ad2-e254-41b4-88cb-e7336ed02860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:57:42.840369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe19ad2-e254-41b4-88cb-e7336ed02860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:42.842474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe19ad2-e254-41b4-88cb-e7336ed02860 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:42.856074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1065/2133] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:57:42.860468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414525-b1bb-4470-a383-f71676366379 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:42.870235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed414525-b1bb-4470-a383-f71676366379 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:42.875201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd20453-4813-4d92-bffc-2892402edc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:42.875971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd20453-4813-4d92-bffc-2892402edc5f admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:42.876304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd20453-4813-4d92-bffc-2892402edc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:42.876677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd20453-4813-4d92-bffc-2892402edc5f admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:42.877451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd20453-4813-4d92-bffc-2892402edc5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:42.880050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd20453-4813-4d92-bffc-2892402edc5f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:43.202006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1069/2134] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:57:42 2022] POST /identity/v3/projects => generated 381 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:43.207193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe19ad2-e254-41b4-88cb-e7336ed02860 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:43.217993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe19ad2-e254-41b4-88cb-e7336ed02860 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:43.299812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc9da9-68db-4fee-9721-fc73fcdf1b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:43.301485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc9da9-68db-4fee-9721-fc73fcdf1b31 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:43.301895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc9da9-68db-4fee-9721-fc73fcdf1b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:43.302281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc9da9-68db-4fee-9721-fc73fcdf1b31 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:43.303242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc9da9-68db-4fee-9721-fc73fcdf1b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:43.306809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc9da9-68db-4fee-9721-fc73fcdf1b31 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:43.448691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1066/2135] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:57:43 2022] POST /identity/v3/users => generated 291 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:43.453349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd20453-4813-4d92-bffc-2892402edc5f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:43.463968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd20453-4813-4d92-bffc-2892402edc5f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:43.468955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd8e87-75c3-4003-8ee7-322b6717ff87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:43.469542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd8e87-75c3-4003-8ee7-322b6717ff87 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:43.469925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd8e87-75c3-4003-8ee7-322b6717ff87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:43.470344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd8e87-75c3-4003-8ee7-322b6717ff87 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:43.471094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd8e87-75c3-4003-8ee7-322b6717ff87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:43.472705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd8e87-75c3-4003-8ee7-322b6717ff87 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:43.491745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc9da9-68db-4fee-9721-fc73fcdf1b31 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:43.505431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1070/2136] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:43.506485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc9da9-68db-4fee-9721-fc73fcdf1b31 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:43.513118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad65015-c05b-41e5-84c0-7670924fb205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:43.514522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad65015-c05b-41e5-84c0-7670924fb205 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:43.514840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad65015-c05b-41e5-84c0-7670924fb205 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:43.515192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad65015-c05b-41e5-84c0-7670924fb205 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:43.515946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad65015-c05b-41e5-84c0-7670924fb205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:43.518673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad65015-c05b-41e5-84c0-7670924fb205 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:43.542149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1067/2137] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:43.543868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd8e87-75c3-4003-8ee7-322b6717ff87 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:43.562028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd8e87-75c3-4003-8ee7-322b6717ff87 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:43.568207 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af900890-91d3-4bac-b25f-2d0870f2af62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:43.570182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af900890-91d3-4bac-b25f-2d0870f2af62 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:43.570324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af900890-91d3-4bac-b25f-2d0870f2af62 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:43.570586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af900890-91d3-4bac-b25f-2d0870f2af62 admin admin] PATH_INFO: `/v3/projects/c0c1c61746a94cfdb67c9e69787c4656/users/18a669acafbc49dfb3d1fb2e9017a204/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:43.589958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af900890-91d3-4bac-b25f-2d0870f2af62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0c1c61746a94cfdb67c9e69787c4656, user_id=18a669acafbc49dfb3d1fb2e9017a204, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:43.593865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af900890-91d3-4bac-b25f-2d0870f2af62 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:43.745050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1071/2138] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:57:43 2022] PUT /identity/v3/projects/c0c1c61746a94cfdb67c9e69787c4656/users/18a669acafbc49dfb3d1fb2e9017a204/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:43.755689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589a560-895f-4461-af49-822c8f53483e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:43.756211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589a560-895f-4461-af49-822c8f53483e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:43.756323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589a560-895f-4461-af49-822c8f53483e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:43.807230 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-4589a560-895f-4461-af49-822c8f53483e None None] MFA Rules not processed for user `18a669acafbc49dfb3d1fb2e9017a204`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:43.810647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4589a560-895f-4461-af49-822c8f53483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:43.906342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1068/2139] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:43.911345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af900890-91d3-4bac-b25f-2d0870f2af62 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:43.935487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:44.002905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af900890-91d3-4bac-b25f-2d0870f2af62 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:44.008472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0519c-4b71-4883-871b-a4fffc490ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:44.009174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0519c-4b71-4883-871b-a4fffc490ffd admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:44.009606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0519c-4b71-4883-871b-a4fffc490ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:44.009932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0519c-4b71-4883-871b-a4fffc490ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:44.011047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0519c-4b71-4883-871b-a4fffc490ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:44.012629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0519c-4b71-4883-871b-a4fffc490ffd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:44.112725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1072/2140] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:57:43 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:44.113858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589a560-895f-4461-af49-822c8f53483e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:44.130898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589a560-895f-4461-af49-822c8f53483e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:44.240079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3dfe4-0f16-418f-834f-6a301f0e0991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:44.240902 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3dfe4-0f16-418f-834f-6a301f0e0991 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:44.241915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3dfe4-0f16-418f-834f-6a301f0e0991 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:44.241915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3dfe4-0f16-418f-834f-6a301f0e0991 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:44.242571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3dfe4-0f16-418f-834f-6a301f0e0991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:44.244599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3dfe4-0f16-418f-834f-6a301f0e0991 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:44.255017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:44.328931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1069/2141] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:57:44 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:44.333857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0519c-4b71-4883-871b-a4fffc490ffd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:44.357213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0519c-4b71-4883-871b-a4fffc490ffd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:44.362825 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4485d-a4ff-43c2-b1db-6c8ab63069e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:44.363756 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4485d-a4ff-43c2-b1db-6c8ab63069e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:44.364221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4485d-a4ff-43c2-b1db-6c8ab63069e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:44.364639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4485d-a4ff-43c2-b1db-6c8ab63069e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:44.365641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4485d-a4ff-43c2-b1db-6c8ab63069e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:44.367418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4485d-a4ff-43c2-b1db-6c8ab63069e0 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:44.385863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1073/2142] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:44.389530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3dfe4-0f16-418f-834f-6a301f0e0991 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:44.401015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3dfe4-0f16-418f-834f-6a301f0e0991 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:44.408450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9898bbe-aea6-4ff6-b5e6-59401201c75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:44.409456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9898bbe-aea6-4ff6-b5e6-59401201c75b admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:44.410006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9898bbe-aea6-4ff6-b5e6-59401201c75b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:44.410391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9898bbe-aea6-4ff6-b5e6-59401201c75b admin admin] PATH_INFO: `/v3/projects/32381fc22bad430a9f3ea9e7590be06c/users/936496c4f2764d64b42e5cd175462dfc/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:44.435514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9898bbe-aea6-4ff6-b5e6-59401201c75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=32381fc22bad430a9f3ea9e7590be06c, user_id=936496c4f2764d64b42e5cd175462dfc, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:44.439327 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9898bbe-aea6-4ff6-b5e6-59401201c75b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:44.524155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1070/2143] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:57:44 2022] PUT /identity/v3/projects/32381fc22bad430a9f3ea9e7590be06c/users/936496c4f2764d64b42e5cd175462dfc/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:44.527660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4485d-a4ff-43c2-b1db-6c8ab63069e0 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:44.543983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4485d-a4ff-43c2-b1db-6c8ab63069e0 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:44.655202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff39ee-cd76-4774-b7c0-c685180b894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:44.655975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff39ee-cd76-4774-b7c0-c685180b894d admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:44.656335 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff39ee-cd76-4774-b7c0-c685180b894d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:44.657134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff39ee-cd76-4774-b7c0-c685180b894d admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:44.657980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff39ee-cd76-4774-b7c0-c685180b894d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:44.661872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff39ee-cd76-4774-b7c0-c685180b894d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:44.675904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1074/2144] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:44.679820 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9898bbe-aea6-4ff6-b5e6-59401201c75b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:44.698222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9898bbe-aea6-4ff6-b5e6-59401201c75b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:44.721852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14d4f2-b315-4d13-8002-fd45c6113332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:44.724053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14d4f2-b315-4d13-8002-fd45c6113332 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:44.724448 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14d4f2-b315-4d13-8002-fd45c6113332 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:44.724873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14d4f2-b315-4d13-8002-fd45c6113332 admin admin] PATH_INFO: `/v3/projects/32381fc22bad430a9f3ea9e7590be06c/users/936496c4f2764d64b42e5cd175462dfc/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:44.727318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14d4f2-b315-4d13-8002-fd45c6113332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32381fc22bad430a9f3ea9e7590be06c, user_id=936496c4f2764d64b42e5cd175462dfc, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:44.730530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14d4f2-b315-4d13-8002-fd45c6113332 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:44.751051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1071/2145] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:57:44 2022] PUT /identity/v3/projects/32381fc22bad430a9f3ea9e7590be06c/users/936496c4f2764d64b42e5cd175462dfc/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:44.761090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eda199-c1e8-4b43-86ce-7b33f2e3ca40 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:44.761806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eda199-c1e8-4b43-86ce-7b33f2e3ca40 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:44.762358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eda199-c1e8-4b43-86ce-7b33f2e3ca40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:44.828042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a7eda199-c1e8-4b43-86ce-7b33f2e3ca40 None None] MFA Rules not processed for user `936496c4f2764d64b42e5cd175462dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:44.829564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a7eda199-c1e8-4b43-86ce-7b33f2e3ca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:44.948289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1075/2146] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:44.960638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14d4f2-b315-4d13-8002-fd45c6113332 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:44.962671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0c14d4f2-b315-4d13-8002-fd45c6113332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:45.107565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14d4f2-b315-4d13-8002-fd45c6113332 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:45.116313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4958aa-be80-4fbe-bc8d-a2c4993e06c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:45.119325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4958aa-be80-4fbe-bc8d-a2c4993e06c3 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:45.119747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4958aa-be80-4fbe-bc8d-a2c4993e06c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:45.120108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4958aa-be80-4fbe-bc8d-a2c4993e06c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:45.140475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4958aa-be80-4fbe-bc8d-a2c4993e06c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:45.142539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4958aa-be80-4fbe-bc8d-a2c4993e06c3 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:45.159630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1072/2147] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:46.077139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eda199-c1e8-4b43-86ce-7b33f2e3ca40 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:46.092333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:46.175582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eda199-c1e8-4b43-86ce-7b33f2e3ca40 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:46.188443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:46.190737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:46.193380 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:46.193737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] PATH_INFO: `/v3/users/18a669acafbc49dfb3d1fb2e9017a204` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:46.197323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a669acafbc49dfb3d1fb2e9017a204)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:46.199438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:46.458810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a669acafbc49dfb3d1fb2e9017a204'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:46.562753 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a669acafbc49dfb3d1fb2e9017a204'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:46.563944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a669acafbc49dfb3d1fb2e9017a204'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:46.577640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a669acafbc49dfb3d1fb2e9017a204'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:46.578715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a669acafbc49dfb3d1fb2e9017a204'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:46.626565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1076/2148] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:57:46 2022] DELETE /identity/v3/users/18a669acafbc49dfb3d1fb2e9017a204 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:46.631640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c58c-bfec-4935-ae71-3bf07be8985d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:46.632157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c58c-bfec-4935-ae71-3bf07be8985d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:46.633400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c58c-bfec-4935-ae71-3bf07be8985d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:46.669923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6bd6c58c-bfec-4935-ae71-3bf07be8985d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:46.671496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6bd6c58c-bfec-4935-ae71-3bf07be8985d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:46.794048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1073/2149] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:46.805582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:46.807053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:46.965789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdd570-a339-423f-bdc1-40c00cf34ab0 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:46.981653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38301dfb-b573-4405-a565-7e3d5d78f427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:46.982575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38301dfb-b573-4405-a565-7e3d5d78f427 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:46.982860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38301dfb-b573-4405-a565-7e3d5d78f427 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:46.983140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38301dfb-b573-4405-a565-7e3d5d78f427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:46.994205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38301dfb-b573-4405-a565-7e3d5d78f427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:46.995656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38301dfb-b573-4405-a565-7e3d5d78f427 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:47.018596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1077/2150] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:48.108939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6c58c-bfec-4935-ae71-3bf07be8985d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:48.110472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6bd6c58c-bfec-4935-ae71-3bf07be8985d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:48.125387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6c58c-bfec-4935-ae71-3bf07be8985d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:48.132372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:48.136039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:48.136039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:48.136039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] PATH_INFO: `/v3/projects/c0c1c61746a94cfdb67c9e69787c4656` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:48.136039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0c1c61746a94cfdb67c9e69787c4656)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:48.138591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:48.397191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:48.446734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0c1c61746a94cfdb67c9e69787c4656', 'request_id': 'req-25b36e63-359a-4fbd-9373-eb09baf2e86b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:48.471074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0c1c61746a94cfdb67c9e69787c4656', 'request_id': 'req-25b36e63-359a-4fbd-9373-eb09baf2e86b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:48.564640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0c1c61746a94cfdb67c9e69787c4656', 'request_id': 'req-25b36e63-359a-4fbd-9373-eb09baf2e86b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:48.602771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] The token cache is being invalidate because project c0c1c61746a94cfdb67c9e69787c4656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:57:48.603386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:48.607593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:48.613064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1074/2151] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:57:48 2022] DELETE /identity/v3/projects/c0c1c61746a94cfdb67c9e69787c4656 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:57:48.619956 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38301dfb-b573-4405-a565-7e3d5d78f427 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:48.635616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38301dfb-b573-4405-a565-7e3d5d78f427 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:48.691353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:48.723902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:48.724802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:48.725084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:48.725415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] PATH_INFO: `/v3/users/936496c4f2764d64b42e5cd175462dfc` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:48.726951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] RBAC: Authorizing `identity:delete_user(user_id=936496c4f2764d64b42e5cd175462dfc)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:48.729663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:48.993808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936496c4f2764d64b42e5cd175462dfc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:49.123904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936496c4f2764d64b42e5cd175462dfc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:49.124763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936496c4f2764d64b42e5cd175462dfc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:49.163085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936496c4f2764d64b42e5cd175462dfc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:49.163952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-864bc26c-24c7-4240-b810-58e44384474b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936496c4f2764d64b42e5cd175462dfc'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:49.202977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1078/2152] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:57:48 2022] DELETE /identity/v3/users/936496c4f2764d64b42e5cd175462dfc => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:51.100227 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:51.101741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:51.129671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b36e63-359a-4fbd-9373-eb09baf2e86b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:51.263036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:51.264006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:51.264298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:51.264631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] PATH_INFO: `/v3/projects/32381fc22bad430a9f3ea9e7590be06c` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:51.266303 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32381fc22bad430a9f3ea9e7590be06c)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:51.271234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:51.464561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32381fc22bad430a9f3ea9e7590be06c', 'request_id': 'req-25f96463-2c19-4bdb-91f0-452f5d8b5252'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:51.474564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32381fc22bad430a9f3ea9e7590be06c', 'request_id': 'req-25f96463-2c19-4bdb-91f0-452f5d8b5252'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:51.564330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32381fc22bad430a9f3ea9e7590be06c', 'request_id': 'req-25f96463-2c19-4bdb-91f0-452f5d8b5252'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:51.610667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] The token cache is being invalidate because project 32381fc22bad430a9f3ea9e7590be06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:57:51.612555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:51.614153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-25f96463-2c19-4bdb-91f0-452f5d8b5252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:57:51.617458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1075/2153] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:57:51 2022] DELETE /identity/v3/projects/32381fc22bad430a9f3ea9e7590be06c => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:51.622547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e004bec-96e4-4aaf-8fbf-c895daa422b9 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:51.624860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e004bec-96e4-4aaf-8fbf-c895daa422b9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:51.624860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e004bec-96e4-4aaf-8fbf-c895daa422b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:51.647270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7e004bec-96e4-4aaf-8fbf-c895daa422b9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:51.649848 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7e004bec-96e4-4aaf-8fbf-c895daa422b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:51.765740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1079/2154] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:57:51.766103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb714931-3789-4712-81fc-b9866ec77cb2 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:51.766399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb714931-3789-4712-81fc-b9866ec77cb2 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:51.766680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb714931-3789-4712-81fc-b9866ec77cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:51.807056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-eb714931-3789-4712-81fc-b9866ec77cb2 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:51.808315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-eb714931-3789-4712-81fc-b9866ec77cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:51.829601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1076/2155] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 15:57:51.834870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e004bec-96e4-4aaf-8fbf-c895daa422b9 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:51.855610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e004bec-96e4-4aaf-8fbf-c895daa422b9 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:51.867073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb24a5d-5297-4e4b-97f8-2975703f6f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:51.868767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb24a5d-5297-4e4b-97f8-2975703f6f59 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:51.870126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb24a5d-5297-4e4b-97f8-2975703f6f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:51.871764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb24a5d-5297-4e4b-97f8-2975703f6f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:51.874738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb24a5d-5297-4e4b-97f8-2975703f6f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:57:51.874738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb24a5d-5297-4e4b-97f8-2975703f6f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:51.874738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb24a5d-5297-4e4b-97f8-2975703f6f59 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:51.909627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1080/2156] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:57:51.914082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb714931-3789-4712-81fc-b9866ec77cb2 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:51.930097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb714931-3789-4712-81fc-b9866ec77cb2 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:51.939767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf82e92-15ca-4438-86ab-89d7e9d6bf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:51.940559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf82e92-15ca-4438-86ab-89d7e9d6bf3b admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:51.941061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf82e92-15ca-4438-86ab-89d7e9d6bf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:51.941963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf82e92-15ca-4438-86ab-89d7e9d6bf3b admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:51.943027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf82e92-15ca-4438-86ab-89d7e9d6bf3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:51.947612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf82e92-15ca-4438-86ab-89d7e9d6bf3b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:52.085627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1077/2157] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:57:51 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:52.086939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb24a5d-5297-4e4b-97f8-2975703f6f59 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:52.102097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb24a5d-5297-4e4b-97f8-2975703f6f59 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:52.198647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:52.247137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec7caa-4e6b-45f7-856e-4319f9d5611e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:52.249220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec7caa-4e6b-45f7-856e-4319f9d5611e admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:52.250271 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec7caa-4e6b-45f7-856e-4319f9d5611e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:52.254571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec7caa-4e6b-45f7-856e-4319f9d5611e admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:52.254571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec7caa-4e6b-45f7-856e-4319f9d5611e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:52.264413 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec7caa-4e6b-45f7-856e-4319f9d5611e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:52.436051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1081/2158] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:57:52 2022] POST /identity/v3/users => generated 288 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:57:52.437437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf82e92-15ca-4438-86ab-89d7e9d6bf3b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:52.452211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf82e92-15ca-4438-86ab-89d7e9d6bf3b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:52.458538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ab6b9-df51-41ba-a5ce-c2063cab94d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:52.459417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ab6b9-df51-41ba-a5ce-c2063cab94d4 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:52.459880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ab6b9-df51-41ba-a5ce-c2063cab94d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:52.460314 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ab6b9-df51-41ba-a5ce-c2063cab94d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:52.461382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ab6b9-df51-41ba-a5ce-c2063cab94d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:52.482093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ab6b9-df51-41ba-a5ce-c2063cab94d4 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:52.525942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1078/2159] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:52.530088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec7caa-4e6b-45f7-856e-4319f9d5611e admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:52.545700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec7caa-4e6b-45f7-856e-4319f9d5611e admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:52.557258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d35b0e-d116-409c-bd8e-f18253f4e11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:52.558012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d35b0e-d116-409c-bd8e-f18253f4e11c admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:52.558361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d35b0e-d116-409c-bd8e-f18253f4e11c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:52.558706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d35b0e-d116-409c-bd8e-f18253f4e11c admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:52.559512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d35b0e-d116-409c-bd8e-f18253f4e11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:52.561451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d35b0e-d116-409c-bd8e-f18253f4e11c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:52.591470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ab6b9-df51-41ba-a5ce-c2063cab94d4 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:52.592531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1082/2160] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:57:52.605474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:57:52.610800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ab6b9-df51-41ba-a5ce-c2063cab94d4 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:52.625529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ea45d-0e42-4e75-90b7-5ead75a8be2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:52.626492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ea45d-0e42-4e75-90b7-5ead75a8be2e admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:52.626792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ea45d-0e42-4e75-90b7-5ead75a8be2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:52.627079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ea45d-0e42-4e75-90b7-5ead75a8be2e admin admin] PATH_INFO: `/v3/projects/fe724e7c19ce44aeb2c84b4e3c5e9290/users/c7d08009396541538ae8a3c63cca91f4/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:52.650755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ea45d-0e42-4e75-90b7-5ead75a8be2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe724e7c19ce44aeb2c84b4e3c5e9290, user_id=c7d08009396541538ae8a3c63cca91f4, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:52.654024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ea45d-0e42-4e75-90b7-5ead75a8be2e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:53.097706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1079/2161] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:57:52 2022] PUT /identity/v3/projects/fe724e7c19ce44aeb2c84b4e3c5e9290/users/c7d08009396541538ae8a3c63cca91f4/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:57:53.108332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525db890-d1e5-4a29-8b48-e8e317e951b2 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:53.114865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525db890-d1e5-4a29-8b48-e8e317e951b2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:53.114865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525db890-d1e5-4a29-8b48-e8e317e951b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:53.137661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-525db890-d1e5-4a29-8b48-e8e317e951b2 None None] MFA Rules not processed for user `c7d08009396541538ae8a3c63cca91f4`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:53.143511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-525db890-d1e5-4a29-8b48-e8e317e951b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:53.231100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1083/2162] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:57:53.243972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ea45d-0e42-4e75-90b7-5ead75a8be2e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:57:53.255112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9f5ea45d-0e42-4e75-90b7-5ead75a8be2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:53.464244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ea45d-0e42-4e75-90b7-5ead75a8be2e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:57:53.479513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cbf23-3564-4dd6-84e0-d5fa7209ba57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:57:53.480196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cbf23-3564-4dd6-84e0-d5fa7209ba57 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:53.481340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cbf23-3564-4dd6-84e0-d5fa7209ba57 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:53.482295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cbf23-3564-4dd6-84e0-d5fa7209ba57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:53.504918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cbf23-3564-4dd6-84e0-d5fa7209ba57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:57:53.512282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cbf23-3564-4dd6-84e0-d5fa7209ba57 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:57:53.537417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1080/2163] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:57:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:57:55.770565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d111e60-7918-4b70-a5eb-e6afc7edfe69 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:57:55.771002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d111e60-7918-4b70-a5eb-e6afc7edfe69 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:57:55.771191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d111e60-7918-4b70-a5eb-e6afc7edfe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:57:55.789162 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-3d111e60-7918-4b70-a5eb-e6afc7edfe69 None None] MFA Rules not processed for user `c7d08009396541538ae8a3c63cca91f4`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:57:55.791486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3d111e60-7918-4b70-a5eb-e6afc7edfe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:57:55.797413 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1084/2164] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:57:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:01.880707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cbf23-3564-4dd6-84e0-d5fa7209ba57 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:01.883581 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e43cbf23-3564-4dd6-84e0-d5fa7209ba57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:02.093695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cbf23-3564-4dd6-84e0-d5fa7209ba57 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:02.109233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7911a-b514-469c-bb13-efc633ba4284 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:02.112043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7911a-b514-469c-bb13-efc633ba4284 service placement] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:02.112043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7911a-b514-469c-bb13-efc633ba4284 service placement] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:02.112043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7911a-b514-469c-bb13-efc633ba4284 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:02.114159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-28c7911a-b514-469c-bb13-efc633ba4284 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:02.129578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7911a-b514-469c-bb13-efc633ba4284 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:02.132285 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7911a-b514-469c-bb13-efc633ba4284 service placement] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:02.150881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1081/2165] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:58:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:58:05.241286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dc1ab-dbfd-4d38-b775-7f7dc6924d6e None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:05.241751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dc1ab-dbfd-4d38-b775-7f7dc6924d6e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:05.242205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dc1ab-dbfd-4d38-b775-7f7dc6924d6e None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:05.243806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1085/2166] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:58:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:58:05.249639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee17210-e649-44f2-8be3-af7d30bd05b6 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:05.250255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee17210-e649-44f2-8be3-af7d30bd05b6 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:05.250377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee17210-e649-44f2-8be3-af7d30bd05b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:05.270393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-fee17210-e649-44f2-8be3-af7d30bd05b6 None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:05.271639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fee17210-e649-44f2-8be3-af7d30bd05b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:05.344634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1082/2167] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:05.356742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dc1ab-dbfd-4d38-b775-7f7dc6924d6e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:05.358412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-239dc1ab-dbfd-4d38-b775-7f7dc6924d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:05.437991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dc1ab-dbfd-4d38-b775-7f7dc6924d6e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:05.442130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ac762-c007-48d0-8e68-06112f6a8e27 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5eeda3264191409eba35ffbb908770a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:05.443268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ac762-c007-48d0-8e68-06112f6a8e27 service glance] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:05.443268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ac762-c007-48d0-8e68-06112f6a8e27 service glance] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:05.443510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ac762-c007-48d0-8e68-06112f6a8e27 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:05.454046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ac762-c007-48d0-8e68-06112f6a8e27 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:05.455481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ac762-c007-48d0-8e68-06112f6a8e27 service glance] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:05.466966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1086/2168] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:58:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:58:05.673588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80aeec-d5cd-4b99-bd6f-6cb8223f3d71 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:05.673914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80aeec-d5cd-4b99-bd6f-6cb8223f3d71 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:05.674200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80aeec-d5cd-4b99-bd6f-6cb8223f3d71 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:05.675855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1083/2169] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:58:05.680694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb6927-1591-4d5a-ad64-e500e5f5c149 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:05.680959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb6927-1591-4d5a-ad64-e500e5f5c149 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:05.681238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb6927-1591-4d5a-ad64-e500e5f5c149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:05.699941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-29fb6927-1591-4d5a-ad64-e500e5f5c149 None None] MFA Rules not processed for user `7e63b8e907764f2c818391ab2ad9184c`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:05.701157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-29fb6927-1591-4d5a-ad64-e500e5f5c149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:05.720466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:05.753483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1087/2170] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:05.766255 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80aeec-d5cd-4b99-bd6f-6cb8223f3d71 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:05.767985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9f80aeec-d5cd-4b99-bd6f-6cb8223f3d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:05.776754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:05.781325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80aeec-d5cd-4b99-bd6f-6cb8223f3d71 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:05.784906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279431d-1e02-4fa5-a509-ec338fa9ff97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:05.785554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279431d-1e02-4fa5-a509-ec338fa9ff97 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:05.785867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279431d-1e02-4fa5-a509-ec338fa9ff97 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:05.786154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279431d-1e02-4fa5-a509-ec338fa9ff97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:05.793215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279431d-1e02-4fa5-a509-ec338fa9ff97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:05.794721 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279431d-1e02-4fa5-a509-ec338fa9ff97 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:05.803281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1084/2171] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:58:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:58:05.915329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ef8e-b800-4e6b-8ae9-1fde93e03712 None None] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:05.915628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ef8e-b800-4e6b-8ae9-1fde93e03712 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:05.915912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40ef8e-b800-4e6b-8ae9-1fde93e03712 None None] PATH_INFO: `/` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:05.917405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1088/2172] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 15:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 15:58:05.923011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b0ef-d8b4-499d-a471-acf4c7a6173e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:05.923302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b0ef-d8b4-499d-a471-acf4c7a6173e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:05.923580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5b0ef-d8b4-499d-a471-acf4c7a6173e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:05.945215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ada5b0ef-d8b4-499d-a471-acf4c7a6173e None None] MFA Rules not processed for user `ce88c973feef4627a1b871ee9cb26511`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:05.946414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ada5b0ef-d8b4-499d-a471-acf4c7a6173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:05.956367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1085/2173] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:05.967050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40ef8e-b800-4e6b-8ae9-1fde93e03712 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:05.969313 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5b40ef8e-b800-4e6b-8ae9-1fde93e03712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:05.984261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40ef8e-b800-4e6b-8ae9-1fde93e03712 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:05.987900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70df9ed-a330-42eb-a707-db25b2d8bbb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:05.989056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70df9ed-a330-42eb-a707-db25b2d8bbb1 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:05.989342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70df9ed-a330-42eb-a707-db25b2d8bbb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:05.989628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70df9ed-a330-42eb-a707-db25b2d8bbb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:06.000986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70df9ed-a330-42eb-a707-db25b2d8bbb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:06.002242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70df9ed-a330-42eb-a707-db25b2d8bbb1 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:06.015738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1089/2174] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:58:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:58:07.682814 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62197f9-2459-476f-a8be-eaa6787550b4 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:07.683387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62197f9-2459-476f-a8be-eaa6787550b4 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:07.683484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62197f9-2459-476f-a8be-eaa6787550b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:07.703770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d62197f9-2459-476f-a8be-eaa6787550b4 None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:07.705270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d62197f9-2459-476f-a8be-eaa6787550b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:07.775769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1086/2175] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:58:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:07.790605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70df9ed-a330-42eb-a707-db25b2d8bbb1 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:07.793200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e70df9ed-a330-42eb-a707-db25b2d8bbb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:07.873719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70df9ed-a330-42eb-a707-db25b2d8bbb1 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:07.888453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b52fe-9693-44e3-a60c-70248d7b5238 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:07.889459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b52fe-9693-44e3-a60c-70248d7b5238 service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:07.889969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b52fe-9693-44e3-a60c-70248d7b5238 service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:07.890562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b52fe-9693-44e3-a60c-70248d7b5238 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:07.902186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b52fe-9693-44e3-a60c-70248d7b5238 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:07.903987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b52fe-9693-44e3-a60c-70248d7b5238 service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:07.915805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1090/2176] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:58:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:58:15.726725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d808b0c-08d9-498b-9ce2-adcf18753d2b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:15.727789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d808b0c-08d9-498b-9ce2-adcf18753d2b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:15.728072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d808b0c-08d9-498b-9ce2-adcf18753d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:15.755642 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5d808b0c-08d9-498b-9ce2-adcf18753d2b None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:15.759420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5d808b0c-08d9-498b-9ce2-adcf18753d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:15.767702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1087/2177] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:58:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:15.785504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b52fe-9693-44e3-a60c-70248d7b5238 service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:15.800064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b52fe-9693-44e3-a60c-70248d7b5238 service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:15.806935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429d528-98a6-46e9-bf16-8944fd51f0b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:15.807799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429d528-98a6-46e9-bf16-8944fd51f0b0 service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:15.808208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429d528-98a6-46e9-bf16-8944fd51f0b0 service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:15.808605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429d528-98a6-46e9-bf16-8944fd51f0b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:15.822039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429d528-98a6-46e9-bf16-8944fd51f0b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:15.824152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429d528-98a6-46e9-bf16-8944fd51f0b0 service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:15.837024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1091/2178] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:58:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:58:25.749207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=35 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:25.830896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=56 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:28.950149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70711b-f41e-46b7-a96c-39dc8558c08b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:28.950683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70711b-f41e-46b7-a96c-39dc8558c08b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:28.950796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70711b-f41e-46b7-a96c-39dc8558c08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:28.969992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0e70711b-f41e-46b7-a96c-39dc8558c08b None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:28.971198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0e70711b-f41e-46b7-a96c-39dc8558c08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:28.979025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1088/2179] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 14 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:28.993328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429d528-98a6-46e9-bf16-8944fd51f0b0 service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:29.004225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429d528-98a6-46e9-bf16-8944fd51f0b0 service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:29.009594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537f71-6a99-424d-ba4f-f417d39a1344 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:29.010404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537f71-6a99-424d-ba4f-f417d39a1344 service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:29.010810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537f71-6a99-424d-ba4f-f417d39a1344 service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:29.011099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537f71-6a99-424d-ba4f-f417d39a1344 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:29.021724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39537f71-6a99-424d-ba4f-f417d39a1344 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:29.023356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39537f71-6a99-424d-ba4f-f417d39a1344 service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:29.032940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1092/2180] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:58:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:58:32.372049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70711b-f41e-46b7-a96c-39dc8558c08b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:32.374553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0e70711b-f41e-46b7-a96c-39dc8558c08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:32.443210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70711b-f41e-46b7-a96c-39dc8558c08b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:32.447692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:32.448565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:32.448851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:32.449129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] PATH_INFO: `/v3/users/a84d1283721f408d85c0b835d9173daf` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:32.450366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84d1283721f408d85c0b835d9173daf)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:32.452324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:32.623200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84d1283721f408d85c0b835d9173daf'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:32.711533 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84d1283721f408d85c0b835d9173daf'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:32.712192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84d1283721f408d85c0b835d9173daf'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:32.719736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84d1283721f408d85c0b835d9173daf'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:32.720369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8bd10d-828d-4f91-8f5b-23b9e30af3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84d1283721f408d85c0b835d9173daf'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:32.738867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1089/2181] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:58:32 2022] DELETE /identity/v3/users/a84d1283721f408d85c0b835d9173daf => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:58:33.015924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537f71-6a99-424d-ba4f-f417d39a1344 service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:33.031786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537f71-6a99-424d-ba4f-f417d39a1344 service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:33.115435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:33.116035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:33.116526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:33.116888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] PATH_INFO: `/v3/projects/8edf005aaa274e98a3f098e45105c0da` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:33.118229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8edf005aaa274e98a3f098e45105c0da)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:33.120260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:33.234594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8edf005aaa274e98a3f098e45105c0da', 'request_id': 'req-84831192-ea9c-4734-a17d-29ee75b0ce50'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:33.245945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8edf005aaa274e98a3f098e45105c0da', 'request_id': 'req-84831192-ea9c-4734-a17d-29ee75b0ce50'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:33.316782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8edf005aaa274e98a3f098e45105c0da', 'request_id': 'req-84831192-ea9c-4734-a17d-29ee75b0ce50'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:33.338972 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] The token cache is being invalidate because project 8edf005aaa274e98a3f098e45105c0da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:58:33.339495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:33.341397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84831192-ea9c-4734-a17d-29ee75b0ce50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:33.343286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1093/2182] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:58:33 2022] DELETE /identity/v3/projects/8edf005aaa274e98a3f098e45105c0da => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:58:33.351534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31427d3f-042a-407a-baca-070be139228d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:33.352048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31427d3f-042a-407a-baca-070be139228d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:33.352561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31427d3f-042a-407a-baca-070be139228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:33.380976 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-31427d3f-042a-407a-baca-070be139228d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:33.382923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-31427d3f-042a-407a-baca-070be139228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:33.484336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1090/2183] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:33.497277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b81305-58e6-445d-ace0-cf355de7af3a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:33.497681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b81305-58e6-445d-ace0-cf355de7af3a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:33.498112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b81305-58e6-445d-ace0-cf355de7af3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:33.519586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-51b81305-58e6-445d-ace0-cf355de7af3a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:33.520843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-51b81305-58e6-445d-ace0-cf355de7af3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:33.529308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1094/2184] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:33.533713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31427d3f-042a-407a-baca-070be139228d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:33.544136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31427d3f-042a-407a-baca-070be139228d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:33.549083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d941d-5cdb-4fc8-b64e-163b5855f6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:33.550586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d941d-5cdb-4fc8-b64e-163b5855f6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:33.550890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d941d-5cdb-4fc8-b64e-163b5855f6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:33.551181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d941d-5cdb-4fc8-b64e-163b5855f6c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:33.551892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d941d-5cdb-4fc8-b64e-163b5855f6c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:58:33.552196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d941d-5cdb-4fc8-b64e-163b5855f6c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:33.553923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d941d-5cdb-4fc8-b64e-163b5855f6c2 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:33.563060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:33.567149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1091/2185] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:58:33.571792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b81305-58e6-445d-ace0-cf355de7af3a None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:33.580993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b81305-58e6-445d-ace0-cf355de7af3a None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:33.586205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facb6d1-7e6b-4c5b-85f3-d0058348eb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:33.586868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facb6d1-7e6b-4c5b-85f3-d0058348eb81 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:33.587152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facb6d1-7e6b-4c5b-85f3-d0058348eb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:33.587431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4facb6d1-7e6b-4c5b-85f3-d0058348eb81 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:33.588134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facb6d1-7e6b-4c5b-85f3-d0058348eb81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:33.590312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4facb6d1-7e6b-4c5b-85f3-d0058348eb81 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:33.712142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1095/2186] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:58:33 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:58:33.716617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d941d-5cdb-4fc8-b64e-163b5855f6c2 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:33.726663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d941d-5cdb-4fc8-b64e-163b5855f6c2 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:33.792401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b360dc-396f-401e-acfa-1792f1ada7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:33.793090 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b360dc-396f-401e-acfa-1792f1ada7be admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:33.793369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b360dc-396f-401e-acfa-1792f1ada7be admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:33.793647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b360dc-396f-401e-acfa-1792f1ada7be admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:33.794395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b360dc-396f-401e-acfa-1792f1ada7be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:33.796372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b360dc-396f-401e-acfa-1792f1ada7be admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:33.882081 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1092/2187] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:58:33 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:58:33.886877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facb6d1-7e6b-4c5b-85f3-d0058348eb81 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:33.898497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4facb6d1-7e6b-4c5b-85f3-d0058348eb81 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:33.905623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30b510-8627-436c-8436-4049802467f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:33.906686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30b510-8627-436c-8436-4049802467f1 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:33.907143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30b510-8627-436c-8436-4049802467f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:33.907531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30b510-8627-436c-8436-4049802467f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:33.908813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30b510-8627-436c-8436-4049802467f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:33.912331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30b510-8627-436c-8436-4049802467f1 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:33.932135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1096/2188] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:58:33.936772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b360dc-396f-401e-acfa-1792f1ada7be admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:33.951883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b360dc-396f-401e-acfa-1792f1ada7be admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:33.956611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443303b3-88a4-4730-adc0-e6a8c7f97535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:33.957236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443303b3-88a4-4730-adc0-e6a8c7f97535 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:33.957526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443303b3-88a4-4730-adc0-e6a8c7f97535 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:33.957911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443303b3-88a4-4730-adc0-e6a8c7f97535 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:33.958640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443303b3-88a4-4730-adc0-e6a8c7f97535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:33.960294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443303b3-88a4-4730-adc0-e6a8c7f97535 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:33.974563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1093/2189] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:58:33.979792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30b510-8627-436c-8436-4049802467f1 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:33.996354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30b510-8627-436c-8436-4049802467f1 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:34.007266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f44abc-0f9c-42fb-b475-b48bebb1b1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:34.007266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f44abc-0f9c-42fb-b475-b48bebb1b1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:34.007266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f44abc-0f9c-42fb-b475-b48bebb1b1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:34.007266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f44abc-0f9c-42fb-b475-b48bebb1b1bd admin admin] PATH_INFO: `/v3/projects/181e18e1cce84f33a53c24d1e56f883b/users/ed44804de41442aa98f1a2fa8143c673/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:34.026973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f44abc-0f9c-42fb-b475-b48bebb1b1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=181e18e1cce84f33a53c24d1e56f883b, user_id=ed44804de41442aa98f1a2fa8143c673, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:34.030628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f44abc-0f9c-42fb-b475-b48bebb1b1bd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:34.121948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1097/2190] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:58:33 2022] PUT /identity/v3/projects/181e18e1cce84f33a53c24d1e56f883b/users/ed44804de41442aa98f1a2fa8143c673/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:58:34.187631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915c21-9a0d-470c-8f73-da0d4225f6fb None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:34.187930 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915c21-9a0d-470c-8f73-da0d4225f6fb None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:34.188211 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d915c21-9a0d-470c-8f73-da0d4225f6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:34.226284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-3d915c21-9a0d-470c-8f73-da0d4225f6fb None None] MFA Rules not processed for user `ed44804de41442aa98f1a2fa8143c673`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:34.227425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3d915c21-9a0d-470c-8f73-da0d4225f6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:34.300889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1094/2191] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:58:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:34.313248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f44abc-0f9c-42fb-b475-b48bebb1b1bd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:34.316089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a0f44abc-0f9c-42fb-b475-b48bebb1b1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:34.337053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:34.387304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f44abc-0f9c-42fb-b475-b48bebb1b1bd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:34.391118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb569d1-9cd1-42c7-bd07-8d572e27df2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5eeda3264191409eba35ffbb908770a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:34.391778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb569d1-9cd1-42c7-bd07-8d572e27df2c service glance] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:34.392199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb569d1-9cd1-42c7-bd07-8d572e27df2c service glance] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:34.392488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb569d1-9cd1-42c7-bd07-8d572e27df2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:34.402132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb569d1-9cd1-42c7-bd07-8d572e27df2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:34.403669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb569d1-9cd1-42c7-bd07-8d572e27df2c service glance] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:34.414228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1098/2192] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:58:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:58:44.297369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:44.780238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=20 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:50.281795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9cddd-b493-4a37-b157-b1575200182b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:50.282416 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9cddd-b493-4a37-b157-b1575200182b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:50.282627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9cddd-b493-4a37-b157-b1575200182b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:50.303687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-40c9cddd-b493-4a37-b157-b1575200182b None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:50.304833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-40c9cddd-b493-4a37-b157-b1575200182b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:50.366072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1095/2193] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 14 15:58:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:50.389158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb569d1-9cd1-42c7-bd07-8d572e27df2c service glance] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:50.392529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ddb569d1-9cd1-42c7-bd07-8d572e27df2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:50.468034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb569d1-9cd1-42c7-bd07-8d572e27df2c service glance] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:50.473075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fbdbb-d857-4b18-b454-698fffa8f82f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:50.473856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fbdbb-d857-4b18-b454-698fffa8f82f service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:50.474155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fbdbb-d857-4b18-b454-698fffa8f82f service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:50.474435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fbdbb-d857-4b18-b454-698fffa8f82f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:50.483195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fbdbb-d857-4b18-b454-698fffa8f82f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:50.484809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fbdbb-d857-4b18-b454-698fffa8f82f service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:50.493756 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1099/2194] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:58:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:58:52.220359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff8477-3455-4f1b-a3a9-42cbb26fef0d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:52.220878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff8477-3455-4f1b-a3a9-42cbb26fef0d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:52.220986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff8477-3455-4f1b-a3a9-42cbb26fef0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:52.240726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-73ff8477-3455-4f1b-a3a9-42cbb26fef0d None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:52.241902 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-73ff8477-3455-4f1b-a3a9-42cbb26fef0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:52.249233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1096/2195] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:58:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:52.261235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fbdbb-d857-4b18-b454-698fffa8f82f service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:52.272784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fbdbb-d857-4b18-b454-698fffa8f82f service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:52.277940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048893b-8a59-47c3-bfb9-350859286447 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:52.278623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048893b-8a59-47c3-bfb9-350859286447 service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:52.278907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048893b-8a59-47c3-bfb9-350859286447 service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:52.279186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048893b-8a59-47c3-bfb9-350859286447 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:52.288433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048893b-8a59-47c3-bfb9-350859286447 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:52.290070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048893b-8a59-47c3-bfb9-350859286447 service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:52.299015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1100/2196] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:58:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 15:58:53.402703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf301046-b93c-43df-ad0d-50529a4e5935 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:53.403190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf301046-b93c-43df-ad0d-50529a4e5935 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:53.403421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf301046-b93c-43df-ad0d-50529a4e5935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:53.423025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-bf301046-b93c-43df-ad0d-50529a4e5935 None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:53.424376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-bf301046-b93c-43df-ad0d-50529a4e5935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:53.434861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1097/2197] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 14 15:58:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:53.448492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048893b-8a59-47c3-bfb9-350859286447 service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:53.459546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048893b-8a59-47c3-bfb9-350859286447 service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:53.464918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64de79a-3698-447d-b2b0-871d7e99ffef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:53.465683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64de79a-3698-447d-b2b0-871d7e99ffef service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:53.466072 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64de79a-3698-447d-b2b0-871d7e99ffef service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:53.466422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64de79a-3698-447d-b2b0-871d7e99ffef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:53.476109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64de79a-3698-447d-b2b0-871d7e99ffef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:53.477922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64de79a-3698-447d-b2b0-871d7e99ffef service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:53.486755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1101/2198] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:58:54.245751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fc6ef-4222-404a-ba5d-7c8a8b316136 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:54.246056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fc6ef-4222-404a-ba5d-7c8a8b316136 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:54.246330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fc6ef-4222-404a-ba5d-7c8a8b316136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:54.264435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f23fc6ef-4222-404a-ba5d-7c8a8b316136 None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:54.265629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f23fc6ef-4222-404a-ba5d-7c8a8b316136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:54.273796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1098/2199] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 15:58:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:54.291424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64de79a-3698-447d-b2b0-871d7e99ffef service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:54.296812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64de79a-3698-447d-b2b0-871d7e99ffef service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:54.302936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76007e-eebf-4751-a6d8-855e834aaaa1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:54.303601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76007e-eebf-4751-a6d8-855e834aaaa1 service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:54.303884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76007e-eebf-4751-a6d8-855e834aaaa1 service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:54.304156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76007e-eebf-4751-a6d8-855e834aaaa1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:54.313034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76007e-eebf-4751-a6d8-855e834aaaa1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:54.314836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76007e-eebf-4751-a6d8-855e834aaaa1 service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:54.323733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1102/2200] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:58:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 15:58:55.167019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e86f7-d54e-4e28-8053-b4c6af0a6306 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:55.167493 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e86f7-d54e-4e28-8053-b4c6af0a6306 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:55.167632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e86f7-d54e-4e28-8053-b4c6af0a6306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:55.188516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ed6e86f7-d54e-4e28-8053-b4c6af0a6306 None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:55.189726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ed6e86f7-d54e-4e28-8053-b4c6af0a6306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:55.196958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1099/2201] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 14 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:55.209219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76007e-eebf-4751-a6d8-855e834aaaa1 service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:55.219733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76007e-eebf-4751-a6d8-855e834aaaa1 service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:55.228321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a50bfb-a5bc-4202-9e4f-58ea17dc23b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:55.228321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a50bfb-a5bc-4202-9e4f-58ea17dc23b0 service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:55.228321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a50bfb-a5bc-4202-9e4f-58ea17dc23b0 service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:55.228321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a50bfb-a5bc-4202-9e4f-58ea17dc23b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:55.234984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a50bfb-a5bc-4202-9e4f-58ea17dc23b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:55.236623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a50bfb-a5bc-4202-9e4f-58ea17dc23b0 service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:55.245659 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1103/2202] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 15:58:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 15:58:58.119302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e86f7-d54e-4e28-8053-b4c6af0a6306 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:58.120760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ed6e86f7-d54e-4e28-8053-b4c6af0a6306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:58.190058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e86f7-d54e-4e28-8053-b4c6af0a6306 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:58.195312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:58.196189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:58.196474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:58.196794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] PATH_INFO: `/v3/users/c7d08009396541538ae8a3c63cca91f4` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:58.198070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7d08009396541538ae8a3c63cca91f4)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:58.200001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:58.463462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7d08009396541538ae8a3c63cca91f4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:58.591038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7d08009396541538ae8a3c63cca91f4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:58.591734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7d08009396541538ae8a3c63cca91f4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:58.601446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7d08009396541538ae8a3c63cca91f4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:58.602205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7d08009396541538ae8a3c63cca91f4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:58.616412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:58.622502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1100/2203] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:58:58 2022] DELETE /identity/v3/users/c7d08009396541538ae8a3c63cca91f4 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:58:58.626708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2507740-7cd1-4b3d-a41f-84500ffce281 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:58.627210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2507740-7cd1-4b3d-a41f-84500ffce281 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:58.627308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2507740-7cd1-4b3d-a41f-84500ffce281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:58.646345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e2507740-7cd1-4b3d-a41f-84500ffce281 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:58.647655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e2507740-7cd1-4b3d-a41f-84500ffce281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:58.710328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1104/2204] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:58.720196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:58.721626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:58.781934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ce44e-7ff3-4ffb-93bf-03dad374d410 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:58.787224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4b59b-428a-4868-8186-4bbfd817e512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:58.787890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4b59b-428a-4868-8186-4bbfd817e512 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:58.788226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4b59b-428a-4868-8186-4bbfd817e512 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:58.788505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4b59b-428a-4868-8186-4bbfd817e512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:58.798234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4b59b-428a-4868-8186-4bbfd817e512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:58.799932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4b59b-428a-4868-8186-4bbfd817e512 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:58.811447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1101/2205] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:58:59.100609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2507740-7cd1-4b3d-a41f-84500ffce281 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:59.102180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e2507740-7cd1-4b3d-a41f-84500ffce281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:59.116266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2507740-7cd1-4b3d-a41f-84500ffce281 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:59.121204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:59.122107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:59.122394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:59.122674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] PATH_INFO: `/v3/projects/fe724e7c19ce44aeb2c84b4e3c5e9290` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:59.124428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe724e7c19ce44aeb2c84b4e3c5e9290)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:59.126634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:59.225970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:58:59.238073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe724e7c19ce44aeb2c84b4e3c5e9290', 'request_id': 'req-8d4ace87-6e07-41c1-b3e8-91a6849950e9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:59.246219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe724e7c19ce44aeb2c84b4e3c5e9290', 'request_id': 'req-8d4ace87-6e07-41c1-b3e8-91a6849950e9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:59.351171 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe724e7c19ce44aeb2c84b4e3c5e9290', 'request_id': 'req-8d4ace87-6e07-41c1-b3e8-91a6849950e9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:59.362869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] The token cache is being invalidate because project fe724e7c19ce44aeb2c84b4e3c5e9290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:58:59.363161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:59.363801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8d4ace87-6e07-41c1-b3e8-91a6849950e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:58:59.365382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1105/2206] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:58:59 2022] DELETE /identity/v3/projects/fe724e7c19ce44aeb2c84b4e3c5e9290 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:58:59.372037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbcbb2-5232-4410-a707-5ec0bbe375cb None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:59.372575 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbcbb2-5232-4410-a707-5ec0bbe375cb None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:59.372575 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbcbb2-5232-4410-a707-5ec0bbe375cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:59.393279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d4cbcbb2-5232-4410-a707-5ec0bbe375cb None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:59.394546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d4cbcbb2-5232-4410-a707-5ec0bbe375cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:59.460228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1102/2207] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:59.468173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed323184-1595-427d-bef6-b8de638b9dce None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:59.468462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed323184-1595-427d-bef6-b8de638b9dce None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:59.473782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed323184-1595-427d-bef6-b8de638b9dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:59.488543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ed323184-1595-427d-bef6-b8de638b9dce None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:58:59.489784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ed323184-1595-427d-bef6-b8de638b9dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:58:59.497983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1106/2208] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:58:59.507959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbcbb2-5232-4410-a707-5ec0bbe375cb None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:59.517795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbcbb2-5232-4410-a707-5ec0bbe375cb None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:59.522995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db819b-6a58-4238-8723-4c2092071723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:59.523666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db819b-6a58-4238-8723-4c2092071723 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:59.523950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db819b-6a58-4238-8723-4c2092071723 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:59.524231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db819b-6a58-4238-8723-4c2092071723 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:59.524965 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db819b-6a58-4238-8723-4c2092071723 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:58:59.525269 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db819b-6a58-4238-8723-4c2092071723 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:59.526905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db819b-6a58-4238-8723-4c2092071723 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:59.538841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1103/2209] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 15:58:59.543654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed323184-1595-427d-bef6-b8de638b9dce None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:59.553688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed323184-1595-427d-bef6-b8de638b9dce None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:59.558852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a22c3a-012d-488f-a4f1-b70f52d49393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:59.559533 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a22c3a-012d-488f-a4f1-b70f52d49393 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:59.559815 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a22c3a-012d-488f-a4f1-b70f52d49393 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:59.560109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a22c3a-012d-488f-a4f1-b70f52d49393 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:59.560996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a22c3a-012d-488f-a4f1-b70f52d49393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:59.562975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a22c3a-012d-488f-a4f1-b70f52d49393 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:59.736536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1107/2210] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:58:59 2022] POST /identity/v3/projects => generated 379 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:58:59.741069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db819b-6a58-4238-8723-4c2092071723 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:59.753281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db819b-6a58-4238-8723-4c2092071723 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:59.816265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f70ff-8b52-4e8b-94a3-a0daa8dbfcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:59.817111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f70ff-8b52-4e8b-94a3-a0daa8dbfcbb admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:59.817983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f70ff-8b52-4e8b-94a3-a0daa8dbfcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:59.818445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f70ff-8b52-4e8b-94a3-a0daa8dbfcbb admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:59.819262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f70ff-8b52-4e8b-94a3-a0daa8dbfcbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:59.821192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f70ff-8b52-4e8b-94a3-a0daa8dbfcbb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:59.941486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1104/2211] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:58:59 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:58:59.946044 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a22c3a-012d-488f-a4f1-b70f52d49393 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:58:59.958093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a22c3a-012d-488f-a4f1-b70f52d49393 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:58:59.962840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215482e-7d23-4ad7-b317-1d15d81a25b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:58:59.963523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215482e-7d23-4ad7-b317-1d15d81a25b8 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:58:59.963811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215482e-7d23-4ad7-b317-1d15d81a25b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:58:59.964095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215482e-7d23-4ad7-b317-1d15d81a25b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:58:59.964911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215482e-7d23-4ad7-b317-1d15d81a25b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:58:59.966603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215482e-7d23-4ad7-b317-1d15d81a25b8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:58:59.980344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1108/2212] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:58:59.992315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f70ff-8b52-4e8b-94a3-a0daa8dbfcbb admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:00.003272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f70ff-8b52-4e8b-94a3-a0daa8dbfcbb admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:00.008693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08155b9-8244-4ec7-bd37-ae8a7b81a78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:00.009403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08155b9-8244-4ec7-bd37-ae8a7b81a78b admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:00.009690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08155b9-8244-4ec7-bd37-ae8a7b81a78b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:00.010008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08155b9-8244-4ec7-bd37-ae8a7b81a78b admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:00.010749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08155b9-8244-4ec7-bd37-ae8a7b81a78b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:00.013140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08155b9-8244-4ec7-bd37-ae8a7b81a78b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:00.029108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1105/2213] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:59:00.030578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215482e-7d23-4ad7-b317-1d15d81a25b8 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:00.054558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215482e-7d23-4ad7-b317-1d15d81a25b8 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:00.064201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424c26b-5ed6-4bc4-ac04-8689a60d9a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:00.065138 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424c26b-5ed6-4bc4-ac04-8689a60d9a97 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:00.065424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424c26b-5ed6-4bc4-ac04-8689a60d9a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:00.065702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424c26b-5ed6-4bc4-ac04-8689a60d9a97 admin admin] PATH_INFO: `/v3/projects/93233d8a364e44edad9877e1eaa82b4d/users/ba765c1a8c6c449ca699eb7a44dde50b/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:00.082983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424c26b-5ed6-4bc4-ac04-8689a60d9a97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93233d8a364e44edad9877e1eaa82b4d, user_id=ba765c1a8c6c449ca699eb7a44dde50b, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:00.085747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424c26b-5ed6-4bc4-ac04-8689a60d9a97 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:00.174752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1109/2214] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:59:00 2022] PUT /identity/v3/projects/93233d8a364e44edad9877e1eaa82b4d/users/ba765c1a8c6c449ca699eb7a44dde50b/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:59:00.180758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ef8ea-0984-4597-99df-11a4773ce232 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:00.181056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ef8ea-0984-4597-99df-11a4773ce232 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:00.181340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ef8ea-0984-4597-99df-11a4773ce232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:00.194544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:00.201008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ae3ef8ea-0984-4597-99df-11a4773ce232 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:00.202466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ae3ef8ea-0984-4597-99df-11a4773ce232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:00.267787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1106/2215] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:00.277734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424c26b-5ed6-4bc4-ac04-8689a60d9a97 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:00.279618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f424c26b-5ed6-4bc4-ac04-8689a60d9a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:00.344343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424c26b-5ed6-4bc4-ac04-8689a60d9a97 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:00.348140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3898-279b-41f0-85ef-42614b1c8bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:00.348851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3898-279b-41f0-85ef-42614b1c8bbe service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:00.349135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3898-279b-41f0-85ef-42614b1c8bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:00.349416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c3898-279b-41f0-85ef-42614b1c8bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:00.360064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c3898-279b-41f0-85ef-42614b1c8bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:00.361511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c3898-279b-41f0-85ef-42614b1c8bbe service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:00.372004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1110/2216] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:59:06.448267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbfa1e-2a40-444b-96c9-14d36cd523bf None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:06.448808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbfa1e-2a40-444b-96c9-14d36cd523bf None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:06.448808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbfa1e-2a40-444b-96c9-14d36cd523bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:06.479061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f0bbfa1e-2a40-444b-96c9-14d36cd523bf None None] MFA Rules not processed for user `ba765c1a8c6c449ca699eb7a44dde50b`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:06.480514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f0bbfa1e-2a40-444b-96c9-14d36cd523bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:06.573494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1107/2217] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:06.575947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3898-279b-41f0-85ef-42614b1c8bbe service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:06.589527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c3898-279b-41f0-85ef-42614b1c8bbe service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:06.595192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1d83d-4ec7-4df5-a8d2-54caffde0c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:06.596079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1d83d-4ec7-4df5-a8d2-54caffde0c61 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:06.596458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1d83d-4ec7-4df5-a8d2-54caffde0c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:06.596813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1d83d-4ec7-4df5-a8d2-54caffde0c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:06.597519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1d83d-4ec7-4df5-a8d2-54caffde0c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:06.600811 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1d83d-4ec7-4df5-a8d2-54caffde0c61 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:06.723080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1111/2218] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:59:06 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 15:59:06.724639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbfa1e-2a40-444b-96c9-14d36cd523bf None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:06.737195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bbfa1e-2a40-444b-96c9-14d36cd523bf None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:06.813786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b0a74-418d-4dea-8d21-0f54441b8578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:06.814505 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b0a74-418d-4dea-8d21-0f54441b8578 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:06.814931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b0a74-418d-4dea-8d21-0f54441b8578 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:06.815214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b0a74-418d-4dea-8d21-0f54441b8578 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:06.815916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b0a74-418d-4dea-8d21-0f54441b8578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:06.817917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b0a74-418d-4dea-8d21-0f54441b8578 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:06.902053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1108/2219] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:59:06 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:59:06.908641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1d83d-4ec7-4df5-a8d2-54caffde0c61 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:06.919549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1d83d-4ec7-4df5-a8d2-54caffde0c61 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:06.924196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50db429-fb1d-469e-a57e-749691d54fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:06.925115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50db429-fb1d-469e-a57e-749691d54fde admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:06.925409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50db429-fb1d-469e-a57e-749691d54fde admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:06.925694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50db429-fb1d-469e-a57e-749691d54fde admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:06.926460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50db429-fb1d-469e-a57e-749691d54fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:06.928074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50db429-fb1d-469e-a57e-749691d54fde admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:06.943159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1112/2220] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:59:06.949807 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b0a74-418d-4dea-8d21-0f54441b8578 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:06.961590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b0a74-418d-4dea-8d21-0f54441b8578 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:06.966472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f039e34-d5d2-45b5-8396-710c5565afd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:06.967487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f039e34-d5d2-45b5-8396-710c5565afd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:06.967775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f039e34-d5d2-45b5-8396-710c5565afd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:06.968055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f039e34-d5d2-45b5-8396-710c5565afd7 admin admin] PATH_INFO: `/v3/projects/cf8abc30c0cd47c1b6b1c2d35b8f7f9d/users/a22d987d7e914bdaa018d90232f12bdd/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:06.990286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f039e34-d5d2-45b5-8396-710c5565afd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8abc30c0cd47c1b6b1c2d35b8f7f9d, user_id=a22d987d7e914bdaa018d90232f12bdd, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:06.993763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f039e34-d5d2-45b5-8396-710c5565afd7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:07.144523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1109/2221] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:59:06 2022] PUT /identity/v3/projects/cf8abc30c0cd47c1b6b1c2d35b8f7f9d/users/a22d987d7e914bdaa018d90232f12bdd/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:59:07.148485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50db429-fb1d-469e-a57e-749691d54fde admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:07.158595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50db429-fb1d-469e-a57e-749691d54fde admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:07.183969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:07.219215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384dd2e-44bf-49b2-b43e-f59d729e615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:07.219903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384dd2e-44bf-49b2-b43e-f59d729e615a admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:07.220242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384dd2e-44bf-49b2-b43e-f59d729e615a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:07.220529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384dd2e-44bf-49b2-b43e-f59d729e615a admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:07.221272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384dd2e-44bf-49b2-b43e-f59d729e615a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:07.222953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384dd2e-44bf-49b2-b43e-f59d729e615a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:07.235094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1113/2222] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:59:07.240213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f039e34-d5d2-45b5-8396-710c5565afd7 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:07.248196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:07.252010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f039e34-d5d2-45b5-8396-710c5565afd7 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:07.257378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82536af7-1803-46a6-970f-eb42492ad5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:07.258330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82536af7-1803-46a6-970f-eb42492ad5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:07.258617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82536af7-1803-46a6-970f-eb42492ad5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:07.258899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82536af7-1803-46a6-970f-eb42492ad5dc admin admin] PATH_INFO: `/v3/projects/cf8abc30c0cd47c1b6b1c2d35b8f7f9d/users/a22d987d7e914bdaa018d90232f12bdd/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:07.260947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82536af7-1803-46a6-970f-eb42492ad5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8abc30c0cd47c1b6b1c2d35b8f7f9d, user_id=a22d987d7e914bdaa018d90232f12bdd, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:07.263843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82536af7-1803-46a6-970f-eb42492ad5dc admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:07.275626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1110/2223] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:59:07 2022] PUT /identity/v3/projects/cf8abc30c0cd47c1b6b1c2d35b8f7f9d/users/a22d987d7e914bdaa018d90232f12bdd/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:59:13.643035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c449b1-29cb-4aad-8ebb-baf70c0267a4 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:13.643597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c449b1-29cb-4aad-8ebb-baf70c0267a4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:13.643698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c449b1-29cb-4aad-8ebb-baf70c0267a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:13.674676 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-36c449b1-29cb-4aad-8ebb-baf70c0267a4 None None] MFA Rules not processed for user `a22d987d7e914bdaa018d90232f12bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:13.675991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-36c449b1-29cb-4aad-8ebb-baf70c0267a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:13.761943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1114/2224] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:13.778026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82536af7-1803-46a6-970f-eb42492ad5dc admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:13.779572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-82536af7-1803-46a6-970f-eb42492ad5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:13.868046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82536af7-1803-46a6-970f-eb42492ad5dc admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:13.873743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2488f2e-131d-4ed5-b54c-d9ddb95f04c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:13.874458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2488f2e-131d-4ed5-b54c-d9ddb95f04c8 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:13.874770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2488f2e-131d-4ed5-b54c-d9ddb95f04c8 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:13.875048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2488f2e-131d-4ed5-b54c-d9ddb95f04c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:13.947532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2488f2e-131d-4ed5-b54c-d9ddb95f04c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:13.949468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2488f2e-131d-4ed5-b54c-d9ddb95f04c8 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:13.960817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1111/2225] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:59:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:59:18.521229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8951-e747-4d75-9b05-aad729b70013 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:18.521839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8951-e747-4d75-9b05-aad729b70013 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:18.521839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8951-e747-4d75-9b05-aad729b70013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:18.540584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-6acf8951-e747-4d75-9b05-aad729b70013 None None] MFA Rules not processed for user `54fcab272d7946c69aac92f6d3030942`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:18.541806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6acf8951-e747-4d75-9b05-aad729b70013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:18.631588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1115/2226] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:23.963878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=60,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:25.387263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2488f2e-131d-4ed5-b54c-d9ddb95f04c8 service nova] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:25.388774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f2488f2e-131d-4ed5-b54c-d9ddb95f04c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:25.493971 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2488f2e-131d-4ed5-b54c-d9ddb95f04c8 service nova] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:25.498674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:25.499564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:25.499848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:25.500123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] PATH_INFO: `/v3/users/ed44804de41442aa98f1a2fa8143c673` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:25.501455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed44804de41442aa98f1a2fa8143c673)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:25.503418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:25.854407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed44804de41442aa98f1a2fa8143c673'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:26.007940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed44804de41442aa98f1a2fa8143c673'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:26.008599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed44804de41442aa98f1a2fa8143c673'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:26.016506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed44804de41442aa98f1a2fa8143c673'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:26.025171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed44804de41442aa98f1a2fa8143c673'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:26.039439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1112/2227] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:59:25 2022] DELETE /identity/v3/users/ed44804de41442aa98f1a2fa8143c673 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:59:26.043880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcb42d-4f44-4252-bb89-f91ea9edf776 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:26.044722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcb42d-4f44-4252-bb89-f91ea9edf776 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:26.045203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcb42d-4f44-4252-bb89-f91ea9edf776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:26.066144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-defcb42d-4f44-4252-bb89-f91ea9edf776 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:26.068246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-defcb42d-4f44-4252-bb89-f91ea9edf776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:26.125671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:26.146078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1116/2228] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:26.156617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:26.158166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:26.222888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a2093-8603-4b47-b79a-937d82e50063 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:26.226638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b7301-dea5-4ef1-9eab-a2930a6b13c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:26.227308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b7301-dea5-4ef1-9eab-a2930a6b13c1 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:26.227597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b7301-dea5-4ef1-9eab-a2930a6b13c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:26.227866 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b7301-dea5-4ef1-9eab-a2930a6b13c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:26.237754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b7301-dea5-4ef1-9eab-a2930a6b13c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:26.239742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6b7301-dea5-4ef1-9eab-a2930a6b13c1 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:26.251154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1113/2229] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:59:26.811108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcb42d-4f44-4252-bb89-f91ea9edf776 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:26.812551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-defcb42d-4f44-4252-bb89-f91ea9edf776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:26.826080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcb42d-4f44-4252-bb89-f91ea9edf776 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:26.831177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:26.832054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:26.832333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:26.832692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] PATH_INFO: `/v3/projects/181e18e1cce84f33a53c24d1e56f883b` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:26.833986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=181e18e1cce84f33a53c24d1e56f883b)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:26.836755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:26.978977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181e18e1cce84f33a53c24d1e56f883b', 'request_id': 'req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:26.985886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181e18e1cce84f33a53c24d1e56f883b', 'request_id': 'req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:27.122465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181e18e1cce84f33a53c24d1e56f883b', 'request_id': 'req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:27.135427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] The token cache is being invalidate because project 181e18e1cce84f33a53c24d1e56f883b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:59:27.135708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:27.136350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9ee4ec25-c1b6-4fe1-ab97-d531c85bb6a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:27.137916 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1117/2230] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:59:26 2022] DELETE /identity/v3/projects/181e18e1cce84f33a53c24d1e56f883b => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:59:27.143346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ceafe-8eee-4bb0-8ba3-d423cbb9e62d None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:27.143828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ceafe-8eee-4bb0-8ba3-d423cbb9e62d None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:27.143939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ceafe-8eee-4bb0-8ba3-d423cbb9e62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:27.163209 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-187ceafe-8eee-4bb0-8ba3-d423cbb9e62d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:27.164455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-187ceafe-8eee-4bb0-8ba3-d423cbb9e62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:27.183254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:27.228738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1114/2231] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:27.236851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98918c27-38d0-49e1-aaf2-a62a4bac639e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:27.237070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98918c27-38d0-49e1-aaf2-a62a4bac639e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:27.237353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98918c27-38d0-49e1-aaf2-a62a4bac639e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:27.259079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-98918c27-38d0-49e1-aaf2-a62a4bac639e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:27.260373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-98918c27-38d0-49e1-aaf2-a62a4bac639e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:27.270640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1118/2232] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:27.274849 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ceafe-8eee-4bb0-8ba3-d423cbb9e62d None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:27.284364 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ceafe-8eee-4bb0-8ba3-d423cbb9e62d None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:27.289928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998c75b-6242-400b-9b44-08b866f3f621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:27.290691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998c75b-6242-400b-9b44-08b866f3f621 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:27.290987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998c75b-6242-400b-9b44-08b866f3f621 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:27.291287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998c75b-6242-400b-9b44-08b866f3f621 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:27.292027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998c75b-6242-400b-9b44-08b866f3f621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:59:27.292337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998c75b-6242-400b-9b44-08b866f3f621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:27.294041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998c75b-6242-400b-9b44-08b866f3f621 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:27.306924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1115/2233] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:59:27.311744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98918c27-38d0-49e1-aaf2-a62a4bac639e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:27.321158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98918c27-38d0-49e1-aaf2-a62a4bac639e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:27.326674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c4d2e-30f5-4dae-9dc0-40254570a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:27.327785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c4d2e-30f5-4dae-9dc0-40254570a924 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:27.328507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c4d2e-30f5-4dae-9dc0-40254570a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:27.328768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c4d2e-30f5-4dae-9dc0-40254570a924 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:27.330099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c4d2e-30f5-4dae-9dc0-40254570a924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:27.333835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c4d2e-30f5-4dae-9dc0-40254570a924 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:27.456508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1119/2234] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:59:27 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:59:27.461111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998c75b-6242-400b-9b44-08b866f3f621 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:27.471669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998c75b-6242-400b-9b44-08b866f3f621 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:27.530727 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412ed35-0059-4940-8716-6971b7fe8a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:27.531286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412ed35-0059-4940-8716-6971b7fe8a13 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:27.531569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412ed35-0059-4940-8716-6971b7fe8a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:27.531844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412ed35-0059-4940-8716-6971b7fe8a13 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:27.532545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412ed35-0059-4940-8716-6971b7fe8a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:27.534506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412ed35-0059-4940-8716-6971b7fe8a13 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:27.654151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1116/2235] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:59:27 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:59:27.658495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c4d2e-30f5-4dae-9dc0-40254570a924 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:27.669672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c4d2e-30f5-4dae-9dc0-40254570a924 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:27.674746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36943860-ca2d-4bb8-ae10-fdc6eed25125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:27.675422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36943860-ca2d-4bb8-ae10-fdc6eed25125 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:27.675754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36943860-ca2d-4bb8-ae10-fdc6eed25125 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:27.675967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36943860-ca2d-4bb8-ae10-fdc6eed25125 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:27.676725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36943860-ca2d-4bb8-ae10-fdc6eed25125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:27.678357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36943860-ca2d-4bb8-ae10-fdc6eed25125 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:27.690063 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1120/2236] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:59:27.694288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412ed35-0059-4940-8716-6971b7fe8a13 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:27.705532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412ed35-0059-4940-8716-6971b7fe8a13 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:27.712527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505bb22-e920-46e7-8002-8181a222ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:27.713420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505bb22-e920-46e7-8002-8181a222ca71 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:27.713741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505bb22-e920-46e7-8002-8181a222ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:27.714030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505bb22-e920-46e7-8002-8181a222ca71 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:27.714818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505bb22-e920-46e7-8002-8181a222ca71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:27.716596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505bb22-e920-46e7-8002-8181a222ca71 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:27.729363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1117/2237] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 15:59:27.733777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36943860-ca2d-4bb8-ae10-fdc6eed25125 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:27.746901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36943860-ca2d-4bb8-ae10-fdc6eed25125 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:27.751827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276d809-3270-4530-8336-8f4df46efcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:27.752718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276d809-3270-4530-8336-8f4df46efcdc admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:27.752998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276d809-3270-4530-8336-8f4df46efcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:27.753337 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276d809-3270-4530-8336-8f4df46efcdc admin admin] PATH_INFO: `/v3/projects/03ae40d61986482498235373866a4160/users/6b73340ba7584c09a2f3180c18edeee3/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:27.770830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276d809-3270-4530-8336-8f4df46efcdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ae40d61986482498235373866a4160, user_id=6b73340ba7584c09a2f3180c18edeee3, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:27.774563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276d809-3270-4530-8336-8f4df46efcdc admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:27.860097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1121/2238] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:59:27 2022] PUT /identity/v3/projects/03ae40d61986482498235373866a4160/users/6b73340ba7584c09a2f3180c18edeee3/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:59:27.875717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a299-0f15-4695-b2d3-a03dcd94019a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:27.876040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a299-0f15-4695-b2d3-a03dcd94019a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:27.876427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15a299-0f15-4695-b2d3-a03dcd94019a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:27.911174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ea15a299-0f15-4695-b2d3-a03dcd94019a None None] MFA Rules not processed for user `6b73340ba7584c09a2f3180c18edeee3`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:27.912630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ea15a299-0f15-4695-b2d3-a03dcd94019a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:27.980216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1118/2239] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:27.991711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276d809-3270-4530-8336-8f4df46efcdc admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:27.993448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3276d809-3270-4530-8336-8f4df46efcdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:28.055588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276d809-3270-4530-8336-8f4df46efcdc admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:28.059573 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fff16-909f-4550-a1aa-0819b391580a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:28.060370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fff16-909f-4550-a1aa-0819b391580a service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:28.060688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fff16-909f-4550-a1aa-0819b391580a service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:28.061001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fff16-909f-4550-a1aa-0819b391580a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:28.072433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fff16-909f-4550-a1aa-0819b391580a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:28.074181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fff16-909f-4550-a1aa-0819b391580a service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:28.086369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1122/2240] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:59:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:59:31.681358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15a299-0f15-4695-b2d3-a03dcd94019a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:31.703952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:31.762087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15a299-0f15-4695-b2d3-a03dcd94019a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:31.769196 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:31.771008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:31.771469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:31.771914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] PATH_INFO: `/v3/users/6b73340ba7584c09a2f3180c18edeee3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:31.775624 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b73340ba7584c09a2f3180c18edeee3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:31.784246 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:32.276087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b73340ba7584c09a2f3180c18edeee3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:32.507015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b73340ba7584c09a2f3180c18edeee3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:32.507904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b73340ba7584c09a2f3180c18edeee3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:32.517514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b73340ba7584c09a2f3180c18edeee3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:32.518257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b73340ba7584c09a2f3180c18edeee3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:32.538100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1119/2241] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 15:59:31 2022] DELETE /identity/v3/users/6b73340ba7584c09a2f3180c18edeee3 => generated 0 bytes in 857 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 15:59:32.542757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec976ac-352a-470b-9734-6f86ff3317df None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:32.543275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec976ac-352a-470b-9734-6f86ff3317df None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:32.543679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec976ac-352a-470b-9734-6f86ff3317df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:32.562383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7ec976ac-352a-470b-9734-6f86ff3317df None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:32.563793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7ec976ac-352a-470b-9734-6f86ff3317df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:32.575399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:32.669741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1123/2242] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:32.680249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:32.682503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:32.800625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb922-fc14-4f2f-84f2-0a39df557715 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:32.812726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa249b-182b-48ed-aff9-8bf72d71d446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:32.813423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa249b-182b-48ed-aff9-8bf72d71d446 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:32.813689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa249b-182b-48ed-aff9-8bf72d71d446 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:32.814031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa249b-182b-48ed-aff9-8bf72d71d446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:32.840465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa249b-182b-48ed-aff9-8bf72d71d446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:32.842448 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa249b-182b-48ed-aff9-8bf72d71d446 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:32.870263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1120/2243] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:59:33.414243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec976ac-352a-470b-9734-6f86ff3317df None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:33.416539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7ec976ac-352a-470b-9734-6f86ff3317df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:33.435340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec976ac-352a-470b-9734-6f86ff3317df None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:33.440148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:33.440846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:33.441124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:33.441404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] PATH_INFO: `/v3/projects/03ae40d61986482498235373866a4160` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:33.443289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ae40d61986482498235373866a4160)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:33.445345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:33.617594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ae40d61986482498235373866a4160', 'request_id': 'req-348c4e68-7efd-4d06-b359-8541d13c8d4c'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:33.623561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ae40d61986482498235373866a4160', 'request_id': 'req-348c4e68-7efd-4d06-b359-8541d13c8d4c'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:33.758192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ae40d61986482498235373866a4160', 'request_id': 'req-348c4e68-7efd-4d06-b359-8541d13c8d4c'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:33.772303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] The token cache is being invalidate because project 03ae40d61986482498235373866a4160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 15:59:33.772847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:33.773884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-348c4e68-7efd-4d06-b359-8541d13c8d4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 15:59:33.775658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1124/2244] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 15:59:33 2022] DELETE /identity/v3/projects/03ae40d61986482498235373866a4160 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:59:33.780693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65299ee-5e86-4b6f-983e-c8af7e6afdd7 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:33.781179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65299ee-5e86-4b6f-983e-c8af7e6afdd7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:33.781298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65299ee-5e86-4b6f-983e-c8af7e6afdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:33.811252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-c65299ee-5e86-4b6f-983e-c8af7e6afdd7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:33.812572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c65299ee-5e86-4b6f-983e-c8af7e6afdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:33.861655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:33.898948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1121/2245] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:33.909887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf8307-8f28-4bd6-9907-e8b82d7fadbc None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:33.910202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf8307-8f28-4bd6-9907-e8b82d7fadbc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:33.910993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacf8307-8f28-4bd6-9907-e8b82d7fadbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:33.947814 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-eacf8307-8f28-4bd6-9907-e8b82d7fadbc None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:33.947814 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-eacf8307-8f28-4bd6-9907-e8b82d7fadbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:33.948710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1125/2246] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:33.959051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65299ee-5e86-4b6f-983e-c8af7e6afdd7 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:33.964707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65299ee-5e86-4b6f-983e-c8af7e6afdd7 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:33.970529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e974d25-705a-49a4-9aee-36a41db24405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:33.971250 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e974d25-705a-49a4-9aee-36a41db24405 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:33.971755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e974d25-705a-49a4-9aee-36a41db24405 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:33.972229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e974d25-705a-49a4-9aee-36a41db24405 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:33.973736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e974d25-705a-49a4-9aee-36a41db24405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 15:59:33.974249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e974d25-705a-49a4-9aee-36a41db24405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:33.976088 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e974d25-705a-49a4-9aee-36a41db24405 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:33.990644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1122/2247] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 15:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 15:59:34.027651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf8307-8f28-4bd6-9907-e8b82d7fadbc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:34.039922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacf8307-8f28-4bd6-9907-e8b82d7fadbc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:34.046644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fd683-3f90-47c5-9789-8119d0215f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:34.047489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fd683-3f90-47c5-9789-8119d0215f7f admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:34.047801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fd683-3f90-47c5-9789-8119d0215f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:34.048036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fd683-3f90-47c5-9789-8119d0215f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:34.048728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fd683-3f90-47c5-9789-8119d0215f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:34.051144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fd683-3f90-47c5-9789-8119d0215f7f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:34.357334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1126/2248] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 15:59:34 2022] POST /identity/v3/projects => generated 371 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:59:34.362097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e974d25-705a-49a4-9aee-36a41db24405 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:34.372914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e974d25-705a-49a4-9aee-36a41db24405 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:34.432068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68acda4-23c2-4fee-893f-85aae36de8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:34.432774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68acda4-23c2-4fee-893f-85aae36de8e9 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:34.433067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68acda4-23c2-4fee-893f-85aae36de8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:34.433375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68acda4-23c2-4fee-893f-85aae36de8e9 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:34.434123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68acda4-23c2-4fee-893f-85aae36de8e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:34.436172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68acda4-23c2-4fee-893f-85aae36de8e9 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:34.641159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1123/2249] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 15:59:34 2022] POST /identity/v3/users => generated 286 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 15:59:34.645526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fd683-3f90-47c5-9789-8119d0215f7f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:34.656853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fd683-3f90-47c5-9789-8119d0215f7f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:34.661986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e597d-ce2b-4f7e-9c0c-190e59c37b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:34.662655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e597d-ce2b-4f7e-9c0c-190e59c37b76 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:34.663068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e597d-ce2b-4f7e-9c0c-190e59c37b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:34.663341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e597d-ce2b-4f7e-9c0c-190e59c37b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:34.664119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e597d-ce2b-4f7e-9c0c-190e59c37b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:34.665711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e597d-ce2b-4f7e-9c0c-190e59c37b76 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:34.677428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1127/2250] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:59:34.681692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68acda4-23c2-4fee-893f-85aae36de8e9 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:34.691764 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68acda4-23c2-4fee-893f-85aae36de8e9 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:34.696265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49c1de-5477-44a1-b436-e7f75d998060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:34.696918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49c1de-5477-44a1-b436-e7f75d998060 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:34.697199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49c1de-5477-44a1-b436-e7f75d998060 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:34.697481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49c1de-5477-44a1-b436-e7f75d998060 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:34.698248 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49c1de-5477-44a1-b436-e7f75d998060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:34.699857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49c1de-5477-44a1-b436-e7f75d998060 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:34.709911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1124/2251] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 15:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 15:59:34.718436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e597d-ce2b-4f7e-9c0c-190e59c37b76 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:34.729029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e597d-ce2b-4f7e-9c0c-190e59c37b76 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:34.733620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be78b9f-0691-4a56-92a2-e8758b7e971b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:34.734547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be78b9f-0691-4a56-92a2-e8758b7e971b admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:34.734835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be78b9f-0691-4a56-92a2-e8758b7e971b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:34.735114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be78b9f-0691-4a56-92a2-e8758b7e971b admin admin] PATH_INFO: `/v3/projects/0826366f294847ceaed3197605bca80a/users/957e51fee69e4aab80b02ac8eb30c331/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:34.750237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be78b9f-0691-4a56-92a2-e8758b7e971b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0826366f294847ceaed3197605bca80a, user_id=957e51fee69e4aab80b02ac8eb30c331, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:34.752899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be78b9f-0691-4a56-92a2-e8758b7e971b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:34.952023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1128/2252] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 15:59:34 2022] PUT /identity/v3/projects/0826366f294847ceaed3197605bca80a/users/957e51fee69e4aab80b02ac8eb30c331/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 15:59:34.964218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5bebe-6a97-40e2-a80a-cdaf8309223b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:34.964800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5bebe-6a97-40e2-a80a-cdaf8309223b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:34.964800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5bebe-6a97-40e2-a80a-cdaf8309223b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:34.995753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6df5bebe-6a97-40e2-a80a-cdaf8309223b None None] MFA Rules not processed for user `957e51fee69e4aab80b02ac8eb30c331`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:34.996597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6df5bebe-6a97-40e2-a80a-cdaf8309223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:35.096142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1125/2253] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:35.108297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be78b9f-0691-4a56-92a2-e8758b7e971b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:35.110248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4be78b9f-0691-4a56-92a2-e8758b7e971b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:35.175869 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be78b9f-0691-4a56-92a2-e8758b7e971b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:35.179830 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc009800-13d9-47b5-9ce8-48a2aec0e57c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5eeda3264191409eba35ffbb908770a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:35.180483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc009800-13d9-47b5-9ce8-48a2aec0e57c service glance] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:35.180767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc009800-13d9-47b5-9ce8-48a2aec0e57c service glance] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:35.181042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc009800-13d9-47b5-9ce8-48a2aec0e57c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:35.189379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc009800-13d9-47b5-9ce8-48a2aec0e57c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:35.190836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc009800-13d9-47b5-9ce8-48a2aec0e57c service glance] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:35.200350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1129/2254] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:59:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:59:35.803359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:40.511860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb0f66-3128-4ff6-bdf1-ae48048700f2 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:40.512372 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb0f66-3128-4ff6-bdf1-ae48048700f2 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:40.512498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb0f66-3128-4ff6-bdf1-ae48048700f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:40.530734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0cfb0f66-3128-4ff6-bdf1-ae48048700f2 None None] MFA Rules not processed for user `c4fd1155ff11417db8c647f0aadf74a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:40.531947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0cfb0f66-3128-4ff6-bdf1-ae48048700f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:40.568148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:40.594815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1126/2255] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:40.929011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e479c9-28ca-41e4-af85-0ba668cf621e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:40.930437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e479c9-28ca-41e4-af85-0ba668cf621e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:40.930889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e479c9-28ca-41e4-af85-0ba668cf621e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:40.952411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-55e479c9-28ca-41e4-af85-0ba668cf621e None None] MFA Rules not processed for user `ba765c1a8c6c449ca699eb7a44dde50b`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 15:59:40.953788 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-55e479c9-28ca-41e4-af85-0ba668cf621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:41.018656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1130/2256] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 15:59:51.010569 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=35 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:51.018943 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=9 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 15:59:56.481087 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb0f66-3128-4ff6-bdf1-ae48048700f2 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:56.483511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0cfb0f66-3128-4ff6-bdf1-ae48048700f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:56.577684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb0f66-3128-4ff6-bdf1-ae48048700f2 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:56.586927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:56.588018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:56.588546 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:56.588719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:56.591252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:56.689488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:56.690569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:56.704590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1127/2257] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:59:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 15:59:58.791299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e479c9-28ca-41e4-af85-0ba668cf621e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 15:59:58.792923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-55e479c9-28ca-41e4-af85-0ba668cf621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:58.805705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e479c9-28ca-41e4-af85-0ba668cf621e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 15:59:58.809552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a70503-c5f6-4dd2-8b21-85d3379ce50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 15:59:58.810322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a70503-c5f6-4dd2-8b21-85d3379ce50f service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 15:59:58.815232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a70503-c5f6-4dd2-8b21-85d3379ce50f service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 15:59:58.815232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a70503-c5f6-4dd2-8b21-85d3379ce50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 15:59:58.815232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-79a70503-c5f6-4dd2-8b21-85d3379ce50f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 15:59:58.828392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a70503-c5f6-4dd2-8b21-85d3379ce50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 15:59:58.829957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a70503-c5f6-4dd2-8b21-85d3379ce50f service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 15:59:58.842966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1131/2258] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 15:59:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:05.823937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:05.826067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:05.840650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b0d2c-58a7-46ea-9672-05a3c9153f7b service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:05.847125 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67f270-3efe-4877-b21d-6dada55935c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:05.847479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67f270-3efe-4877-b21d-6dada55935c4 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:05.848353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67f270-3efe-4877-b21d-6dada55935c4 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:05.848568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67f270-3efe-4877-b21d-6dada55935c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:05.850434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1d67f270-3efe-4877-b21d-6dada55935c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:05.922873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67f270-3efe-4877-b21d-6dada55935c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:05.924752 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67f270-3efe-4877-b21d-6dada55935c4 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:05.936155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1128/2259] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:08.837239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=6 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:13.883617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a70503-c5f6-4dd2-8b21-85d3379ce50f service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:13.955067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a70503-c5f6-4dd2-8b21-85d3379ce50f service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:13.959706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:13.960590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:13.960876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:13.961703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] PATH_INFO: `/v3/users/957e51fee69e4aab80b02ac8eb30c331` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:13.962558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=957e51fee69e4aab80b02ac8eb30c331)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:13.965126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:14.227236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '957e51fee69e4aab80b02ac8eb30c331'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:14.320100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '957e51fee69e4aab80b02ac8eb30c331'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:14.320841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '957e51fee69e4aab80b02ac8eb30c331'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:14.330016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '957e51fee69e4aab80b02ac8eb30c331'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:14.330780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '957e51fee69e4aab80b02ac8eb30c331'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:14.355109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1132/2260] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:00:13 2022] DELETE /identity/v3/users/957e51fee69e4aab80b02ac8eb30c331 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:00:14.359583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b57a43-0c48-40ab-bff0-8377fd28cfeb None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:14.360065 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b57a43-0c48-40ab-bff0-8377fd28cfeb None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:14.360199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b57a43-0c48-40ab-bff0-8377fd28cfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:14.378274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-43b57a43-0c48-40ab-bff0-8377fd28cfeb None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:00:14.380226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-43b57a43-0c48-40ab-bff0-8377fd28cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:14.419660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:14.441795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1129/2261] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:00:14.452197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:14.459889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:14.516966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee7117-0d30-4b67-81b4-9826753fd531 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:14.520560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb18650-4ffb-4fe0-9cde-a7c271b2a96a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:14.521264 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb18650-4ffb-4fe0-9cde-a7c271b2a96a service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:14.521525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb18650-4ffb-4fe0-9cde-a7c271b2a96a service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:14.521797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb18650-4ffb-4fe0-9cde-a7c271b2a96a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:14.532030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb18650-4ffb-4fe0-9cde-a7c271b2a96a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:14.534053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb18650-4ffb-4fe0-9cde-a7c271b2a96a service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:14.544899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1133/2262] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:15.040742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b57a43-0c48-40ab-bff0-8377fd28cfeb None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:15.043223 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-43b57a43-0c48-40ab-bff0-8377fd28cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:15.061069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b57a43-0c48-40ab-bff0-8377fd28cfeb None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:15.067538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:15.072593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:15.072593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:15.072593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] PATH_INFO: `/v3/projects/0826366f294847ceaed3197605bca80a` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:15.072593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0826366f294847ceaed3197605bca80a)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:15.073412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:15.220135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0826366f294847ceaed3197605bca80a', 'request_id': 'req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:15.231282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0826366f294847ceaed3197605bca80a', 'request_id': 'req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:15.322630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0826366f294847ceaed3197605bca80a', 'request_id': 'req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:15.337693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] The token cache is being invalidate because project 0826366f294847ceaed3197605bca80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:00:15.338299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:15.338985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-00ad1d34-8759-4aa4-a275-fe2f15d21b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:15.341444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1130/2263] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:00:15 2022] DELETE /identity/v3/projects/0826366f294847ceaed3197605bca80a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:00:17.499511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb18650-4ffb-4fe0-9cde-a7c271b2a96a service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:17.500942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0cb18650-4ffb-4fe0-9cde-a7c271b2a96a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:17.529316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:17.588046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb18650-4ffb-4fe0-9cde-a7c271b2a96a service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:17.667196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56618501-92c5-4442-b1d2-6304958aa93d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:17.668375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56618501-92c5-4442-b1d2-6304958aa93d service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:17.668375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56618501-92c5-4442-b1d2-6304958aa93d service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:17.668610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56618501-92c5-4442-b1d2-6304958aa93d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:17.670989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-56618501-92c5-4442-b1d2-6304958aa93d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:17.750511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56618501-92c5-4442-b1d2-6304958aa93d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:17.754015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56618501-92c5-4442-b1d2-6304958aa93d service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:17.765527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1134/2264] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:19.171852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25330b11-39d3-43f2-bae5-49d89afc2aa1 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:19.172771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25330b11-39d3-43f2-bae5-49d89afc2aa1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:19.173068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25330b11-39d3-43f2-bae5-49d89afc2aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:19.193964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-25330b11-39d3-43f2-bae5-49d89afc2aa1 None None] MFA Rules not processed for user `81ca45714ee44a549fd002b4aa13973b`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:00:19.195296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-25330b11-39d3-43f2-bae5-49d89afc2aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:19.203842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1131/2265] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:00:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:00:21.076223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48818b1a-47be-48c6-b357-119ff9d1fd48 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:21.076798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48818b1a-47be-48c6-b357-119ff9d1fd48 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:21.076798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48818b1a-47be-48c6-b357-119ff9d1fd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:21.094348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-48818b1a-47be-48c6-b357-119ff9d1fd48 None None] MFA Rules not processed for user `95c6d7130d6c4ae4b8460807e6820ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:00:21.095562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-48818b1a-47be-48c6-b357-119ff9d1fd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:21.154706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1135/2266] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:00:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:00:21.167055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25330b11-39d3-43f2-bae5-49d89afc2aa1 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:21.168481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-25330b11-39d3-43f2-bae5-49d89afc2aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:21.187267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25330b11-39d3-43f2-bae5-49d89afc2aa1 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:21.193738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:21.194423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:21.194704 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:21.194983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:21.196810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:21.207759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:21.209375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:21.218906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1132/2267] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:21.536042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48818b1a-47be-48c6-b357-119ff9d1fd48 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:21.537481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-48818b1a-47be-48c6-b357-119ff9d1fd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:21.542759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:21.597979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48818b1a-47be-48c6-b357-119ff9d1fd48 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:21.601748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f246a-62f4-486d-9d66-d18227259230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:21.602440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f246a-62f4-486d-9d66-d18227259230 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:21.602727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f246a-62f4-486d-9d66-d18227259230 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:21.603059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f246a-62f4-486d-9d66-d18227259230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:21.604876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-747f246a-62f4-486d-9d66-d18227259230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:21.615465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f246a-62f4-486d-9d66-d18227259230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:21.620764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f246a-62f4-486d-9d66-d18227259230 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:21.629667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1136/2268] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:22.070623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:22.074431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:22.164645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45232d3d-ad0e-478d-b230-186b13965593 service nova] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:22.170608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:22.171420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:22.171716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:22.172021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:22.174140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:22.187341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:22.188946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:22.199314 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1133/2269] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:27.525769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f246a-62f4-486d-9d66-d18227259230 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:27.536115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f246a-62f4-486d-9d66-d18227259230 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:27.540116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:27.540909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:27.541240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:27.541515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:27.543839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:27.556216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:27.557682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:27.568329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1137/2270] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:28.092553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:28.095422 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:28.108521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6263095e-b824-414a-ad21-67014241b9ef service placement] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:28.116564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:28.118593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:28.119124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:28.119598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:28.121888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:28.164665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:28.212890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:28.214668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:28.230402 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1134/2271] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:37.562949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=31 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:38.647195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=15 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:42.834410 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:42.836701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:42.852648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61bfae-6979-4e80-b63c-04b5f3b8fb79 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:42.858571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b35277-92ad-4df1-9820-91f9bfbec279 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:42.859243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b35277-92ad-4df1-9820-91f9bfbec279 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:42.859521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b35277-92ad-4df1-9820-91f9bfbec279 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:42.859801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b35277-92ad-4df1-9820-91f9bfbec279 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:42.861730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-58b35277-92ad-4df1-9820-91f9bfbec279 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:42.875039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b35277-92ad-4df1-9820-91f9bfbec279 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:42.876664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b35277-92ad-4df1-9820-91f9bfbec279 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:42.889017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1138/2272] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:48.510136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:48.511665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:48.581171 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11e4439-9f48-46f6-8a05-dd24678db1fa service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:48.585746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:48.586647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:48.586933 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:48.587238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] PATH_INFO: `/v3/users/54fcab272d7946c69aac92f6d3030942` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:48.588440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] RBAC: Authorizing `identity:delete_user(user_id=54fcab272d7946c69aac92f6d3030942)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:48.590446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:48.781101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54fcab272d7946c69aac92f6d3030942'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:48.850918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54fcab272d7946c69aac92f6d3030942'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:48.851469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54fcab272d7946c69aac92f6d3030942'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:48.859312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54fcab272d7946c69aac92f6d3030942'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:48.859967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-cbe3c987-3144-41f1-98ae-c86c5b31624e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54fcab272d7946c69aac92f6d3030942'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:48.877953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1135/2273] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:00:48 2022] DELETE /identity/v3/users/54fcab272d7946c69aac92f6d3030942 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:00:49.155087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b35277-92ad-4df1-9820-91f9bfbec279 service nova] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:49.166366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b35277-92ad-4df1-9820-91f9bfbec279 service nova] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:49.227762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:49.228631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:49.228911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:49.229185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] PATH_INFO: `/v3/projects/d16ae801ca124180b78f2fce16acc4f7` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:49.230553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d16ae801ca124180b78f2fce16acc4f7)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:49.232506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:49.355733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16ae801ca124180b78f2fce16acc4f7', 'request_id': 'req-1c65c8f2-0755-46fb-b753-415d37755141'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:49.361552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16ae801ca124180b78f2fce16acc4f7', 'request_id': 'req-1c65c8f2-0755-46fb-b753-415d37755141'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:49.427747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16ae801ca124180b78f2fce16acc4f7', 'request_id': 'req-1c65c8f2-0755-46fb-b753-415d37755141'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:49.439581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] The token cache is being invalidate because project d16ae801ca124180b78f2fce16acc4f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:00:49.439867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:49.440492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1c65c8f2-0755-46fb-b753-415d37755141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:00:49.442671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1139/2274] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:00:49 2022] DELETE /identity/v3/projects/d16ae801ca124180b78f2fce16acc4f7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:00:49.447859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbecdee-acec-432c-8659-1d6db8d56d2a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:49.448146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbecdee-acec-432c-8659-1d6db8d56d2a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:49.448420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbecdee-acec-432c-8659-1d6db8d56d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:49.466491 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ecbecdee-acec-432c-8659-1d6db8d56d2a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:00:49.467697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ecbecdee-acec-432c-8659-1d6db8d56d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:49.528929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1136/2275] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:00:49.536823 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ea508-482f-427e-9ae5-0443ac89ca9d None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:49.537108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ea508-482f-427e-9ae5-0443ac89ca9d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:49.537386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ea508-482f-427e-9ae5-0443ac89ca9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:49.558525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-a60ea508-482f-427e-9ae5-0443ac89ca9d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:00:49.559828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a60ea508-482f-427e-9ae5-0443ac89ca9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:49.569014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1140/2276] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:00:49.573403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbecdee-acec-432c-8659-1d6db8d56d2a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:49.583821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbecdee-acec-432c-8659-1d6db8d56d2a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:49.588538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166e16-ba2c-4bbd-ac10-75f795cd63d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:49.589188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166e16-ba2c-4bbd-ac10-75f795cd63d8 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:49.589469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166e16-ba2c-4bbd-ac10-75f795cd63d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:49.589742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166e16-ba2c-4bbd-ac10-75f795cd63d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:49.590498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa166e16-ba2c-4bbd-ac10-75f795cd63d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:00:49.590811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa166e16-ba2c-4bbd-ac10-75f795cd63d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:49.592953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa166e16-ba2c-4bbd-ac10-75f795cd63d8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:49.605179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1137/2277] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:00:49.609610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ea508-482f-427e-9ae5-0443ac89ca9d None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:49.620014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ea508-482f-427e-9ae5-0443ac89ca9d None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:49.624643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6916b1-8a48-4b10-98ce-4f1ff6b1a7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:49.625296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6916b1-8a48-4b10-98ce-4f1ff6b1a7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:49.625581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6916b1-8a48-4b10-98ce-4f1ff6b1a7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:49.625945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6916b1-8a48-4b10-98ce-4f1ff6b1a7e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:49.626640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6916b1-8a48-4b10-98ce-4f1ff6b1a7e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:49.628550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6916b1-8a48-4b10-98ce-4f1ff6b1a7e8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:49.723422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1141/2278] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:00:49 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:00:49.727571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166e16-ba2c-4bbd-ac10-75f795cd63d8 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:49.737682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166e16-ba2c-4bbd-ac10-75f795cd63d8 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:49.742043 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:49.794444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cf898-6a5f-4fbc-9f39-4e5183cc1bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:49.795111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cf898-6a5f-4fbc-9f39-4e5183cc1bad admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:49.795403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cf898-6a5f-4fbc-9f39-4e5183cc1bad admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:49.795677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cf898-6a5f-4fbc-9f39-4e5183cc1bad admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:49.796368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cf898-6a5f-4fbc-9f39-4e5183cc1bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:49.798605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cf898-6a5f-4fbc-9f39-4e5183cc1bad admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:50.007431 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1138/2279] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:00:49 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:00:50.011760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6916b1-8a48-4b10-98ce-4f1ff6b1a7e8 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:50.023088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6916b1-8a48-4b10-98ce-4f1ff6b1a7e8 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:50.027702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbaa41-5998-4787-9017-81d441730e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:50.028416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbaa41-5998-4787-9017-81d441730e2a admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:50.028694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbaa41-5998-4787-9017-81d441730e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:50.028966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbaa41-5998-4787-9017-81d441730e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:50.029991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbaa41-5998-4787-9017-81d441730e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:50.035699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbaa41-5998-4787-9017-81d441730e2a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:50.043606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1142/2280] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:00:50.047659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cf898-6a5f-4fbc-9f39-4e5183cc1bad admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:50.058331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cf898-6a5f-4fbc-9f39-4e5183cc1bad admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:50.063047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497593da-b88f-46da-a796-ecf465c54955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:50.063712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497593da-b88f-46da-a796-ecf465c54955 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:50.064001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497593da-b88f-46da-a796-ecf465c54955 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:50.064286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497593da-b88f-46da-a796-ecf465c54955 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:50.065016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497593da-b88f-46da-a796-ecf465c54955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:50.066696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497593da-b88f-46da-a796-ecf465c54955 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:50.077453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1139/2281] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:00:50.081454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbaa41-5998-4787-9017-81d441730e2a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:50.092484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbaa41-5998-4787-9017-81d441730e2a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:50.097248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc89dc0-fd4e-4af8-94cd-3ece49d4ba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:50.098803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89dc0-fd4e-4af8-94cd-3ece49d4ba3d admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:50.099086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89dc0-fd4e-4af8-94cd-3ece49d4ba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:50.099364 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc89dc0-fd4e-4af8-94cd-3ece49d4ba3d admin admin] PATH_INFO: `/v3/projects/923fdefb5a7246038c012ee78bf00dc2/users/2c1f1c69e35847b9a6323d8dd3863bf7/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:50.116225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc89dc0-fd4e-4af8-94cd-3ece49d4ba3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=923fdefb5a7246038c012ee78bf00dc2, user_id=2c1f1c69e35847b9a6323d8dd3863bf7, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:50.119341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc89dc0-fd4e-4af8-94cd-3ece49d4ba3d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:50.194330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1143/2282] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:00:50 2022] PUT /identity/v3/projects/923fdefb5a7246038c012ee78bf00dc2/users/2c1f1c69e35847b9a6323d8dd3863bf7/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:00:50.199408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c75b8-7d0e-486a-a0f1-eabf25888c83 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:50.199704 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c75b8-7d0e-486a-a0f1-eabf25888c83 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:50.199986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c75b8-7d0e-486a-a0f1-eabf25888c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:50.218760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-939c75b8-7d0e-486a-a0f1-eabf25888c83 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:00:50.220945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-939c75b8-7d0e-486a-a0f1-eabf25888c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:50.284380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1140/2283] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:00:50.294547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc89dc0-fd4e-4af8-94cd-3ece49d4ba3d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:50.296276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-abc89dc0-fd4e-4af8-94cd-3ece49d4ba3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:50.312424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:50.358836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc89dc0-fd4e-4af8-94cd-3ece49d4ba3d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:50.362451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d056d9-f353-4e44-bfbf-dbe234c3e4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:50.363598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d056d9-f353-4e44-bfbf-dbe234c3e4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:50.363885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d056d9-f353-4e44-bfbf-dbe234c3e4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:50.364158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d056d9-f353-4e44-bfbf-dbe234c3e4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:50.374164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d056d9-f353-4e44-bfbf-dbe234c3e4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:50.376077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d056d9-f353-4e44-bfbf-dbe234c3e4a3 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:50.386198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1144/2284] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:54.230598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c75b8-7d0e-486a-a0f1-eabf25888c83 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:54.232204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-939c75b8-7d0e-486a-a0f1-eabf25888c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:54.330041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c75b8-7d0e-486a-a0f1-eabf25888c83 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:54.337682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8080aa-5836-49ff-ad3c-bbbd9bb269a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:54.348588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8080aa-5836-49ff-ad3c-bbbd9bb269a8 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:54.348588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8080aa-5836-49ff-ad3c-bbbd9bb269a8 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:54.348588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8080aa-5836-49ff-ad3c-bbbd9bb269a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:54.348588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-bc8080aa-5836-49ff-ad3c-bbbd9bb269a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:54.370925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:54.420460 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8080aa-5836-49ff-ad3c-bbbd9bb269a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:54.425022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8080aa-5836-49ff-ad3c-bbbd9bb269a8 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:54.439210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1141/2285] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:00:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:00:56.230938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003651-aafb-450a-a5a1-83143c795b01 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:56.231639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003651-aafb-450a-a5a1-83143c795b01 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:56.232650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003651-aafb-450a-a5a1-83143c795b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:56.272603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-76003651-aafb-450a-a5a1-83143c795b01 None None] MFA Rules not processed for user `2c1f1c69e35847b9a6323d8dd3863bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:00:56.273884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-76003651-aafb-450a-a5a1-83143c795b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:00:56.363058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1145/2286] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:00:56.367673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8080aa-5836-49ff-ad3c-bbbd9bb269a8 service nova] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:56.380180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8080aa-5836-49ff-ad3c-bbbd9bb269a8 service nova] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:56.385409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9395a-743b-4daa-be6b-6ba466c3a83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:56.386375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9395a-743b-4daa-be6b-6ba466c3a83b admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:56.386508 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9395a-743b-4daa-be6b-6ba466c3a83b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:56.386745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9395a-743b-4daa-be6b-6ba466c3a83b admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:56.387466 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9395a-743b-4daa-be6b-6ba466c3a83b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:56.389395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9395a-743b-4daa-be6b-6ba466c3a83b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:56.475389 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1142/2287] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:00:56 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:00:56.479434 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76003651-aafb-450a-a5a1-83143c795b01 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:56.491130 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76003651-aafb-450a-a5a1-83143c795b01 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:56.551330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804864c-c314-405f-837d-65d22dd7f577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:56.551951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804864c-c314-405f-837d-65d22dd7f577 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:56.552228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804864c-c314-405f-837d-65d22dd7f577 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:56.552503 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804864c-c314-405f-837d-65d22dd7f577 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:56.553202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804864c-c314-405f-837d-65d22dd7f577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:56.555180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804864c-c314-405f-837d-65d22dd7f577 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:56.648798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1146/2288] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:00:56 2022] POST /identity/v3/users => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:00:56.649667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9395a-743b-4daa-be6b-6ba466c3a83b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:56.661343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9395a-743b-4daa-be6b-6ba466c3a83b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:56.666973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d27462-5492-4ced-9c63-18bfe4044066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:56.668311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d27462-5492-4ced-9c63-18bfe4044066 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:56.668726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d27462-5492-4ced-9c63-18bfe4044066 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:56.669079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d27462-5492-4ced-9c63-18bfe4044066 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:56.669925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d27462-5492-4ced-9c63-18bfe4044066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:56.671873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d27462-5492-4ced-9c63-18bfe4044066 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:56.687885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1143/2289] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:00:56.692798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804864c-c314-405f-837d-65d22dd7f577 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:56.708614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804864c-c314-405f-837d-65d22dd7f577 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:56.713674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1120c-d128-45fb-a419-04997b0de5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:56.714743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1120c-d128-45fb-a419-04997b0de5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:56.714991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1120c-d128-45fb-a419-04997b0de5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:56.715268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1120c-d128-45fb-a419-04997b0de5e0 admin admin] PATH_INFO: `/v3/projects/bf058108de93470289cd511e2b038cd0/users/47ea9a9b8739445183e53ade0278dd17/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:56.732331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1120c-d128-45fb-a419-04997b0de5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf058108de93470289cd511e2b038cd0, user_id=47ea9a9b8739445183e53ade0278dd17, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:56.735063 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1120c-d128-45fb-a419-04997b0de5e0 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:56.808649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1147/2290] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:00:56 2022] PUT /identity/v3/projects/bf058108de93470289cd511e2b038cd0/users/47ea9a9b8739445183e53ade0278dd17/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:00:56.812492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d27462-5492-4ced-9c63-18bfe4044066 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:56.823695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d27462-5492-4ced-9c63-18bfe4044066 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:56.892284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad4899-4a51-4fa3-abb5-50893d78122c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:56.893138 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad4899-4a51-4fa3-abb5-50893d78122c admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:56.893556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad4899-4a51-4fa3-abb5-50893d78122c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:56.893863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad4899-4a51-4fa3-abb5-50893d78122c admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:56.894654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad4899-4a51-4fa3-abb5-50893d78122c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:56.896888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad4899-4a51-4fa3-abb5-50893d78122c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:56.911815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1144/2291] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:00:56.912938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1120c-d128-45fb-a419-04997b0de5e0 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:00:56.924472 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1120c-d128-45fb-a419-04997b0de5e0 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:00:56.930059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf040ee-789b-42ad-a20d-4290a3a71384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:00:56.931064 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf040ee-789b-42ad-a20d-4290a3a71384 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:00:56.931288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf040ee-789b-42ad-a20d-4290a3a71384 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:00:56.931562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf040ee-789b-42ad-a20d-4290a3a71384 admin admin] PATH_INFO: `/v3/projects/bf058108de93470289cd511e2b038cd0/users/47ea9a9b8739445183e53ade0278dd17/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:00:56.933523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf040ee-789b-42ad-a20d-4290a3a71384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf058108de93470289cd511e2b038cd0, user_id=47ea9a9b8739445183e53ade0278dd17, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:00:56.936332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf040ee-789b-42ad-a20d-4290a3a71384 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:00:56.942973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:00:56.948432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1148/2292] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:00:56 2022] PUT /identity/v3/projects/bf058108de93470289cd511e2b038cd0/users/47ea9a9b8739445183e53ade0278dd17/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:01:03.383989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c224787-2212-44dc-af68-bb4a493880b7 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:03.384513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c224787-2212-44dc-af68-bb4a493880b7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:03.384626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c224787-2212-44dc-af68-bb4a493880b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:03.415820 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-2c224787-2212-44dc-af68-bb4a493880b7 None None] MFA Rules not processed for user `47ea9a9b8739445183e53ade0278dd17`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:01:03.417212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2c224787-2212-44dc-af68-bb4a493880b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:03.488644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1145/2293] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:01:03.500457 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf040ee-789b-42ad-a20d-4290a3a71384 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:01:03.502176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7cf040ee-789b-42ad-a20d-4290a3a71384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:03.563283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf040ee-789b-42ad-a20d-4290a3a71384 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:01:03.567639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db383dd0-b1b5-44d5-b78a-cda8b0a8309c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5eeda3264191409eba35ffbb908770a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:01:03.568631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db383dd0-b1b5-44d5-b78a-cda8b0a8309c service glance] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:03.568918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db383dd0-b1b5-44d5-b78a-cda8b0a8309c service glance] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:03.569196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db383dd0-b1b5-44d5-b78a-cda8b0a8309c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:03.630045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db383dd0-b1b5-44d5-b78a-cda8b0a8309c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:01:03.631505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db383dd0-b1b5-44d5-b78a-cda8b0a8309c service glance] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:01:03.640587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1149/2294] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:01:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:01:03.929999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85475c3-a674-433a-a95e-6bcf36fb98cf None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:03.930295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85475c3-a674-433a-a95e-6bcf36fb98cf None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:03.930577 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85475c3-a674-433a-a95e-6bcf36fb98cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:03.948596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d85475c3-a674-433a-a95e-6bcf36fb98cf None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:01:03.949810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d85475c3-a674-433a-a95e-6bcf36fb98cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:03.957162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:01:04.014343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1146/2295] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:01:04.191012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db383dd0-b1b5-44d5-b78a-cda8b0a8309c service glance] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:01:04.192606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-db383dd0-b1b5-44d5-b78a-cda8b0a8309c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:04.267329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db383dd0-b1b5-44d5-b78a-cda8b0a8309c service glance] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:01:04.273325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4b903-88f4-40a8-a2cd-ca42c53bfe76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:01:04.274025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4b903-88f4-40a8-a2cd-ca42c53bfe76 service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:04.274317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4b903-88f4-40a8-a2cd-ca42c53bfe76 service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:04.274611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f4b903-88f4-40a8-a2cd-ca42c53bfe76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:04.284137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4b903-88f4-40a8-a2cd-ca42c53bfe76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:01:04.286401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f4b903-88f4-40a8-a2cd-ca42c53bfe76 service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:01:04.295044 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1150/2296] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 16:01:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 16:01:14.154534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=20 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:01:14.291785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=81 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:01:16.790084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06517a38-f5d7-48d5-b5ef-def5267c60db None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:16.790886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06517a38-f5d7-48d5-b5ef-def5267c60db None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:16.790886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06517a38-f5d7-48d5-b5ef-def5267c60db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:16.810352 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-06517a38-f5d7-48d5-b5ef-def5267c60db None None] MFA Rules not processed for user `2c1f1c69e35847b9a6323d8dd3863bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:01:16.811673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-06517a38-f5d7-48d5-b5ef-def5267c60db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:16.822902 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1147/2297] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:01:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:01:17.157809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4b903-88f4-40a8-a2cd-ca42c53bfe76 service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:01:17.160608 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-47f4b903-88f4-40a8-a2cd-ca42c53bfe76 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:17.247548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f4b903-88f4-40a8-a2cd-ca42c53bfe76 service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:01:17.254667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8ed69-0d23-4fb9-9c82-9e8735f3265a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:01:17.255579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ed69-0d23-4fb9-9c82-9e8735f3265a service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:17.256153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ed69-0d23-4fb9-9c82-9e8735f3265a service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:17.256524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8ed69-0d23-4fb9-9c82-9e8735f3265a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:17.260184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-58c8ed69-0d23-4fb9-9c82-9e8735f3265a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:17.277945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8ed69-0d23-4fb9-9c82-9e8735f3265a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:01:17.280584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8ed69-0d23-4fb9-9c82-9e8735f3265a service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:01:17.298478 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1151/2298] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:01:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:01:20.321857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23268479-cc11-4e74-beba-02d6da9ecb10 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:20.322789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23268479-cc11-4e74-beba-02d6da9ecb10 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:20.322789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23268479-cc11-4e74-beba-02d6da9ecb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:20.341251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-23268479-cc11-4e74-beba-02d6da9ecb10 None None] MFA Rules not processed for user `dcdbbbf1b9944b09898fa8731184c845`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:01:20.343143 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-23268479-cc11-4e74-beba-02d6da9ecb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:20.422501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1148/2299] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:01:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:01:22.472492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe740b-f6c5-4e15-a6c6-c73a65809869 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:22.474075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe740b-f6c5-4e15-a6c6-c73a65809869 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:22.474771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe740b-f6c5-4e15-a6c6-c73a65809869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:22.502008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0fbe740b-f6c5-4e15-a6c6-c73a65809869 None None] MFA Rules not processed for user `c4fd1155ff11417db8c647f0aadf74a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:01:22.503852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0fbe740b-f6c5-4e15-a6c6-c73a65809869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:22.577252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1152/2300] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:01:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:01:24.418974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae5bb-9d6c-4802-bc08-9b7efeccb731 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:24.419525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae5bb-9d6c-4802-bc08-9b7efeccb731 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:24.419657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ae5bb-9d6c-4802-bc08-9b7efeccb731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:24.474987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ed7ae5bb-9d6c-4802-bc08-9b7efeccb731 None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:01:24.476275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ed7ae5bb-9d6c-4802-bc08-9b7efeccb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:24.498631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1149/2301] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:01:24.513466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe740b-f6c5-4e15-a6c6-c73a65809869 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:01:24.546981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe740b-f6c5-4e15-a6c6-c73a65809869 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:01:24.569397 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca6ce-3937-4c98-ac6e-6ba79fd4814d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:01:24.570939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca6ce-3937-4c98-ac6e-6ba79fd4814d service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:24.571411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca6ce-3937-4c98-ac6e-6ba79fd4814d service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:24.571849 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca6ce-3937-4c98-ac6e-6ba79fd4814d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:24.584858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ca6ce-3937-4c98-ac6e-6ba79fd4814d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:01:24.587384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ca6ce-3937-4c98-ac6e-6ba79fd4814d service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:01:24.598195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1153/2302] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 16:01:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 16:01:34.457720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=31 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:01:34.594465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=57 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:01:43.382342 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ae5bb-9d6c-4802-bc08-9b7efeccb731 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:01:43.384777 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ed7ae5bb-9d6c-4802-bc08-9b7efeccb731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:43.522726 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ae5bb-9d6c-4802-bc08-9b7efeccb731 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:01:43.534251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3446a9-573e-4c64-8ff8-725c58bdc870 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:01:43.534251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3446a9-573e-4c64-8ff8-725c58bdc870 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:43.534251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3446a9-573e-4c64-8ff8-725c58bdc870 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:43.534251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3446a9-573e-4c64-8ff8-725c58bdc870 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:43.535367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-db3446a9-573e-4c64-8ff8-725c58bdc870 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:43.553906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3446a9-573e-4c64-8ff8-725c58bdc870 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:01:43.553906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3446a9-573e-4c64-8ff8-725c58bdc870 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:01:43.566903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1150/2303] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:01:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:01:51.992703 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca6ce-3937-4c98-ac6e-6ba79fd4814d service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:01:52.002992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca6ce-3937-4c98-ac6e-6ba79fd4814d service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:01:52.007431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fceb2-feeb-4f1f-8137-8a82ab51d68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:01:52.008102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fceb2-feeb-4f1f-8137-8a82ab51d68b service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:52.008386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fceb2-feeb-4f1f-8137-8a82ab51d68b service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:52.008665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fceb2-feeb-4f1f-8137-8a82ab51d68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:52.010801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8f3fceb2-feeb-4f1f-8137-8a82ab51d68b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:52.115626 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3fceb2-feeb-4f1f-8137-8a82ab51d68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:01:52.117880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3fceb2-feeb-4f1f-8137-8a82ab51d68b service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:01:52.143634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1154/2304] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:01:53.564103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=34 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:01:54.100835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb62080-7427-4c7b-9731-0309a45d68fa None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:54.101125 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb62080-7427-4c7b-9731-0309a45d68fa None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:54.101607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb62080-7427-4c7b-9731-0309a45d68fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:54.137421 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-2eb62080-7427-4c7b-9731-0309a45d68fa None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:01:54.140157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2eb62080-7427-4c7b-9731-0309a45d68fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:54.175640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fceb2-feeb-4f1f-8137-8a82ab51d68b service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:01:54.186429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1151/2305] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 16:01:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:01:54.197724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3fceb2-feeb-4f1f-8137-8a82ab51d68b service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:01:54.207862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c7020-edbf-41cf-a335-d7aab25b9028 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:01:54.208556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c7020-edbf-41cf-a335-d7aab25b9028 service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:54.208843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c7020-edbf-41cf-a335-d7aab25b9028 service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:54.209567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c7020-edbf-41cf-a335-d7aab25b9028 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:54.236881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c7020-edbf-41cf-a335-d7aab25b9028 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:01:54.251142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c7020-edbf-41cf-a335-d7aab25b9028 service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:01:54.274289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1155/2306] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 16:01:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 16:01:58.960214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557827d1-479f-4b65-a2aa-550c6ef81d20 None None] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:58.960848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557827d1-479f-4b65-a2aa-550c6ef81d20 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:58.960955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557827d1-479f-4b65-a2aa-550c6ef81d20 None None] PATH_INFO: `/` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:58.962701 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1152/2307] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 14 16:01:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 16:01:58.989901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dd31d-4108-401a-9bfe-bb7dae251b46 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:58.990561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dd31d-4108-401a-9bfe-bb7dae251b46 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:58.990759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dd31d-4108-401a-9bfe-bb7dae251b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:59.012509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-021dd31d-4108-401a-9bfe-bb7dae251b46 None None] MFA Rules not processed for user `97524081e59f4775b83d82edf737810e`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:01:59.013981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-021dd31d-4108-401a-9bfe-bb7dae251b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:59.106549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1156/2308] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 14 16:01:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:01:59.124679 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557827d1-479f-4b65-a2aa-550c6ef81d20 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:01:59.126617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-557827d1-479f-4b65-a2aa-550c6ef81d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:59.144408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557827d1-479f-4b65-a2aa-550c6ef81d20 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:01:59.149408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0c0f-7523-4c04-8ab9-ed8a34a3ac9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:01:59.150115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0c0f-7523-4c04-8ab9-ed8a34a3ac9a service placement] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:59.150398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0c0f-7523-4c04-8ab9-ed8a34a3ac9a service placement] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:59.150685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b0c0f-7523-4c04-8ab9-ed8a34a3ac9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:59.160600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b0c0f-7523-4c04-8ab9-ed8a34a3ac9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:01:59.162243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b0c0f-7523-4c04-8ab9-ed8a34a3ac9a service placement] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:01:59.173714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1153/2309] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:01:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:01:59.425875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dd31d-4108-401a-9bfe-bb7dae251b46 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:01:59.427467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-021dd31d-4108-401a-9bfe-bb7dae251b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:01:59.441380 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dd31d-4108-401a-9bfe-bb7dae251b46 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:01:59.447095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:01:59.447976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:01:59.448261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:01:59.448575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] PATH_INFO: `/v3/users/81ca45714ee44a549fd002b4aa13973b` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:01:59.449881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ca45714ee44a549fd002b4aa13973b)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:01:59.451881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:01:59.656748 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ca45714ee44a549fd002b4aa13973b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:01:59.731332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ca45714ee44a549fd002b4aa13973b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:01:59.732281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ca45714ee44a549fd002b4aa13973b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:01:59.745584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ca45714ee44a549fd002b4aa13973b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:01:59.746559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fc1af065-7d4b-443e-9e70-cff36e5dc1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ca45714ee44a549fd002b4aa13973b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:01:59.763501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:01:59.778658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1157/2310] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:01:59 2022] DELETE /identity/v3/users/81ca45714ee44a549fd002b4aa13973b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:00.168931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0c0f-7523-4c04-8ab9-ed8a34a3ac9a service placement] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:00.187870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b0c0f-7523-4c04-8ab9-ed8a34a3ac9a service placement] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:00.372992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:00.376289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:00.378507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:00.380085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] PATH_INFO: `/v3/projects/508fd5c51f544a3db6d0b87e37750e01` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:00.381925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=508fd5c51f544a3db6d0b87e37750e01)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:00.384231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:00.528076 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '508fd5c51f544a3db6d0b87e37750e01', 'request_id': 'req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:00.535133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '508fd5c51f544a3db6d0b87e37750e01', 'request_id': 'req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:00.617941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '508fd5c51f544a3db6d0b87e37750e01', 'request_id': 'req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:00.636687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] The token cache is being invalidate because project 508fd5c51f544a3db6d0b87e37750e01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:02:00.636979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:00.637656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5d1552ea-6d73-4a83-aa88-94d6f7b47d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:00.644537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1154/2311] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:02:00 2022] DELETE /identity/v3/projects/508fd5c51f544a3db6d0b87e37750e01 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:00.646761 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80653b3-3800-4b86-a5b7-0bd6acc9c587 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:00.647429 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80653b3-3800-4b86-a5b7-0bd6acc9c587 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:00.647429 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80653b3-3800-4b86-a5b7-0bd6acc9c587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:00.667512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f80653b3-3800-4b86-a5b7-0bd6acc9c587 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:00.669730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f80653b3-3800-4b86-a5b7-0bd6acc9c587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:00.759320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1158/2312] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:00.776892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a348c-aa42-4d6c-b558-c91ed4c5ae22 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:00.777164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a348c-aa42-4d6c-b558-c91ed4c5ae22 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:00.777440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a348c-aa42-4d6c-b558-c91ed4c5ae22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:00.799108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-9d0a348c-aa42-4d6c-b558-c91ed4c5ae22 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:00.800901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9d0a348c-aa42-4d6c-b558-c91ed4c5ae22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:00.818471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1155/2313] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:02:00.822670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80653b3-3800-4b86-a5b7-0bd6acc9c587 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:00.837239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80653b3-3800-4b86-a5b7-0bd6acc9c587 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:00.845597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd520257-c582-46a8-ae12-406f21de46e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:00.847144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520257-c582-46a8-ae12-406f21de46e3 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:00.847599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520257-c582-46a8-ae12-406f21de46e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:00.848039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd520257-c582-46a8-ae12-406f21de46e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:00.849253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd520257-c582-46a8-ae12-406f21de46e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:02:00.849740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd520257-c582-46a8-ae12-406f21de46e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:00.851541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd520257-c582-46a8-ae12-406f21de46e3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:00.872680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1159/2314] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:02:00.877598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a348c-aa42-4d6c-b558-c91ed4c5ae22 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:00.888791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a348c-aa42-4d6c-b558-c91ed4c5ae22 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:00.895096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92a023-6d7c-4856-a8c0-bb6be584353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:00.895993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92a023-6d7c-4856-a8c0-bb6be584353b admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:00.896334 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92a023-6d7c-4856-a8c0-bb6be584353b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:00.896753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92a023-6d7c-4856-a8c0-bb6be584353b admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:00.897587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92a023-6d7c-4856-a8c0-bb6be584353b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:00.899828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92a023-6d7c-4856-a8c0-bb6be584353b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:01.061602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1156/2315] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:02:00 2022] POST /identity/v3/projects => generated 367 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:01.066087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd520257-c582-46a8-ae12-406f21de46e3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:01.080933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd520257-c582-46a8-ae12-406f21de46e3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:01.162789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad209c-783b-443d-97da-93a721fb40c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:01.163876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad209c-783b-443d-97da-93a721fb40c9 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:01.164377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad209c-783b-443d-97da-93a721fb40c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:01.164723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad209c-783b-443d-97da-93a721fb40c9 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:01.165736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad209c-783b-443d-97da-93a721fb40c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:01.168584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad209c-783b-443d-97da-93a721fb40c9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:01.252970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1160/2316] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:02:01 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:02:01.254182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92a023-6d7c-4856-a8c0-bb6be584353b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:01.265497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92a023-6d7c-4856-a8c0-bb6be584353b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:01.275765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9c5c-b459-4d5e-ba80-5c34c8f36ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:01.276622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9c5c-b459-4d5e-ba80-5c34c8f36ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:01.303173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9c5c-b459-4d5e-ba80-5c34c8f36ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:01.305112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9c5c-b459-4d5e-ba80-5c34c8f36ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:01.306108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a9c5c-b459-4d5e-ba80-5c34c8f36ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:01.308387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a9c5c-b459-4d5e-ba80-5c34c8f36ce8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:01.338740 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1157/2317] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:01.342985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad209c-783b-443d-97da-93a721fb40c9 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:01.356270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad209c-783b-443d-97da-93a721fb40c9 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:01.366639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3f580-6f13-4f8d-92c8-cf4e608840f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:01.367480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3f580-6f13-4f8d-92c8-cf4e608840f9 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:01.367868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3f580-6f13-4f8d-92c8-cf4e608840f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:01.368246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3f580-6f13-4f8d-92c8-cf4e608840f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:01.371172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3f580-6f13-4f8d-92c8-cf4e608840f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:01.373159 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3f580-6f13-4f8d-92c8-cf4e608840f9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:01.389182 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1161/2318] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:01.390861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9c5c-b459-4d5e-ba80-5c34c8f36ce8 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:01.402776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9c5c-b459-4d5e-ba80-5c34c8f36ce8 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:01.407902 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a709b5-0c4d-4896-bb72-2658850f94eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:01.408999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a709b5-0c4d-4896-bb72-2658850f94eb admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:01.409923 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a709b5-0c4d-4896-bb72-2658850f94eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:01.410496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a709b5-0c4d-4896-bb72-2658850f94eb admin admin] PATH_INFO: `/v3/projects/6250e9dbfa3347c495ff35f151054c7e/users/a8eb6a1eb7d74680adf81060fdc2b54f/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:01.436392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a709b5-0c4d-4896-bb72-2658850f94eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6250e9dbfa3347c495ff35f151054c7e, user_id=a8eb6a1eb7d74680adf81060fdc2b54f, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:01.439479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a709b5-0c4d-4896-bb72-2658850f94eb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:01.534200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1158/2319] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:01 2022] PUT /identity/v3/projects/6250e9dbfa3347c495ff35f151054c7e/users/a8eb6a1eb7d74680adf81060fdc2b54f/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:02:01.551245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4416f-d43f-4ad3-bf20-85b9be321689 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:01.551535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4416f-d43f-4ad3-bf20-85b9be321689 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:01.551865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4416f-d43f-4ad3-bf20-85b9be321689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:01.589280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-faa4416f-d43f-4ad3-bf20-85b9be321689 None None] MFA Rules not processed for user `a8eb6a1eb7d74680adf81060fdc2b54f`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:01.590857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-faa4416f-d43f-4ad3-bf20-85b9be321689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:01.663847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:01.713910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1162/2320] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:01.715989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a709b5-0c4d-4896-bb72-2658850f94eb admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:01.720364 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-90a709b5-0c4d-4896-bb72-2658850f94eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:01.736961 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4416f-d43f-4ad3-bf20-85b9be321689 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:01.740657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:01.740827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-faa4416f-d43f-4ad3-bf20-85b9be321689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:01.825228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4416f-d43f-4ad3-bf20-85b9be321689 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:01.832421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a957ea-6dd2-46e8-8942-60ba323f77e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:01.834612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a957ea-6dd2-46e8-8942-60ba323f77e7 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:01.834612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a957ea-6dd2-46e8-8942-60ba323f77e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:01.834612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a957ea-6dd2-46e8-8942-60ba323f77e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:01.839222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a709b5-0c4d-4896-bb72-2658850f94eb admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:01.844477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb19966-d261-4068-ac59-94f89ddc5cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:01.845261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb19966-d261-4068-ac59-94f89ddc5cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:01.847195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e6a957ea-6dd2-46e8-8942-60ba323f77e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:01.848241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb19966-d261-4068-ac59-94f89ddc5cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:01.848534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb19966-d261-4068-ac59-94f89ddc5cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:01.862657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb19966-d261-4068-ac59-94f89ddc5cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:01.864197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb19966-d261-4068-ac59-94f89ddc5cd7 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:01.894627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1159/2321] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:02.017451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a957ea-6dd2-46e8-8942-60ba323f77e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:02.022893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a957ea-6dd2-46e8-8942-60ba323f77e7 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:02.094308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1163/2322] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:05.588972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb19966-d261-4068-ac59-94f89ddc5cd7 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:05.590841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fdb19966-d261-4068-ac59-94f89ddc5cd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:05.728599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb19966-d261-4068-ac59-94f89ddc5cd7 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:05.738413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2a20a-7273-4ac8-9e99-afd369b8a795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:05.739772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a20a-7273-4ac8-9e99-afd369b8a795 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:05.740443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a20a-7273-4ac8-9e99-afd369b8a795 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:05.741891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2a20a-7273-4ac8-9e99-afd369b8a795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:05.744898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2cd2a20a-7273-4ac8-9e99-afd369b8a795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:05.803848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2a20a-7273-4ac8-9e99-afd369b8a795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:05.805410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd2a20a-7273-4ac8-9e99-afd369b8a795 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:05.838423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1160/2323] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:12.061099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=57 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:13.778505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a957ea-6dd2-46e8-8942-60ba323f77e7 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:13.851231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a957ea-6dd2-46e8-8942-60ba323f77e7 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:13.855977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:13.856964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:13.857416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:13.857767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] PATH_INFO: `/v3/users/a8eb6a1eb7d74680adf81060fdc2b54f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:13.859124 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8eb6a1eb7d74680adf81060fdc2b54f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:13.861563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:14.080562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8eb6a1eb7d74680adf81060fdc2b54f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:14.159390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8eb6a1eb7d74680adf81060fdc2b54f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:14.160100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8eb6a1eb7d74680adf81060fdc2b54f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:14.169318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8eb6a1eb7d74680adf81060fdc2b54f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:14.170050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8eb6a1eb7d74680adf81060fdc2b54f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:14.191700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1164/2324] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:02:13 2022] DELETE /identity/v3/users/a8eb6a1eb7d74680adf81060fdc2b54f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:14.196742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93904b79-f780-402c-8b4a-b15840699ccc None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:14.197275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93904b79-f780-402c-8b4a-b15840699ccc None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:14.197411 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93904b79-f780-402c-8b4a-b15840699ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:14.222813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-93904b79-f780-402c-8b4a-b15840699ccc None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:14.224561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-93904b79-f780-402c-8b4a-b15840699ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:14.292841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1161/2325] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:14.307235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:14.308846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:14.385913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de0a31-dd5f-40c8-a01c-677faa4afa89 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:14.391673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb29af6-64bc-4570-afab-938d93b6b6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:14.392710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb29af6-64bc-4570-afab-938d93b6b6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:14.393104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb29af6-64bc-4570-afab-938d93b6b6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:14.393571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb29af6-64bc-4570-afab-938d93b6b6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:14.404933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb29af6-64bc-4570-afab-938d93b6b6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:14.406428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb29af6-64bc-4570-afab-938d93b6b6f2 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:14.418792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1165/2326] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:14.872259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93904b79-f780-402c-8b4a-b15840699ccc None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:14.875718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-93904b79-f780-402c-8b4a-b15840699ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:14.889373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93904b79-f780-402c-8b4a-b15840699ccc None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:14.900116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:14.901870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:14.902425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:14.902723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] PATH_INFO: `/v3/projects/6250e9dbfa3347c495ff35f151054c7e` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:14.904153 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6250e9dbfa3347c495ff35f151054c7e)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:14.906633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:15.051424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:15.065892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6250e9dbfa3347c495ff35f151054c7e', 'request_id': 'req-09c312e2-2008-434c-aa3a-cec237237ea0'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:15.072558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6250e9dbfa3347c495ff35f151054c7e', 'request_id': 'req-09c312e2-2008-434c-aa3a-cec237237ea0'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:15.142708 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6250e9dbfa3347c495ff35f151054c7e', 'request_id': 'req-09c312e2-2008-434c-aa3a-cec237237ea0'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:15.157668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] The token cache is being invalidate because project 6250e9dbfa3347c495ff35f151054c7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:02:15.158093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:15.158672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c312e2-2008-434c-aa3a-cec237237ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:15.161073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1162/2327] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:02:14 2022] DELETE /identity/v3/projects/6250e9dbfa3347c495ff35f151054c7e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:15.168303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48b241-892c-440e-b0e6-0ee21d8ab2e8 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:15.168303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48b241-892c-440e-b0e6-0ee21d8ab2e8 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:15.168303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48b241-892c-440e-b0e6-0ee21d8ab2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:15.187163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ad48b241-892c-440e-b0e6-0ee21d8ab2e8 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:15.188398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ad48b241-892c-440e-b0e6-0ee21d8ab2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:15.206317 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:15.254892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1166/2328] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:15.263713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983316a-845f-4484-9139-8c92f93ffed7 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:15.264278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983316a-845f-4484-9139-8c92f93ffed7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:15.264367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983316a-845f-4484-9139-8c92f93ffed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:15.284368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-2983316a-845f-4484-9139-8c92f93ffed7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:15.285616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2983316a-845f-4484-9139-8c92f93ffed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:15.294052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1163/2329] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:02:15.298955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48b241-892c-440e-b0e6-0ee21d8ab2e8 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:15.312545 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad48b241-892c-440e-b0e6-0ee21d8ab2e8 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:15.321060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50206d-d5c9-467e-8e63-0f8e9bad82fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:15.322167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50206d-d5c9-467e-8e63-0f8e9bad82fc admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:15.322621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50206d-d5c9-467e-8e63-0f8e9bad82fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:15.323077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50206d-d5c9-467e-8e63-0f8e9bad82fc admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:15.324318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50206d-d5c9-467e-8e63-0f8e9bad82fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:02:15.324882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50206d-d5c9-467e-8e63-0f8e9bad82fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:15.327764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50206d-d5c9-467e-8e63-0f8e9bad82fc admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:15.348092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1167/2330] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:02:15.352768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983316a-845f-4484-9139-8c92f93ffed7 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:15.362529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983316a-845f-4484-9139-8c92f93ffed7 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:15.367783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0687ae-fc65-4f6a-867d-32861cafd49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:15.368555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0687ae-fc65-4f6a-867d-32861cafd49f admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:15.368833 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0687ae-fc65-4f6a-867d-32861cafd49f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:15.369113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0687ae-fc65-4f6a-867d-32861cafd49f admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:15.369861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0687ae-fc65-4f6a-867d-32861cafd49f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:15.371862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0687ae-fc65-4f6a-867d-32861cafd49f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:15.497543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1164/2331] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:02:15 2022] POST /identity/v3/projects => generated 357 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:15.498399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50206d-d5c9-467e-8e63-0f8e9bad82fc admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:15.509687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50206d-d5c9-467e-8e63-0f8e9bad82fc admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:15.570042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d6cde-a773-427a-9c1f-f18df134257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:15.570741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d6cde-a773-427a-9c1f-f18df134257e admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:15.571027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d6cde-a773-427a-9c1f-f18df134257e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:15.571308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d6cde-a773-427a-9c1f-f18df134257e admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:15.572715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d6cde-a773-427a-9c1f-f18df134257e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:15.575240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d6cde-a773-427a-9c1f-f18df134257e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:15.724441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1168/2332] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:02:15 2022] POST /identity/v3/users => generated 279 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:02:15.728618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0687ae-fc65-4f6a-867d-32861cafd49f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:15.741177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0687ae-fc65-4f6a-867d-32861cafd49f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:15.746092 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52cb30-a53f-40d4-9399-d3ec46b5dd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:15.746885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52cb30-a53f-40d4-9399-d3ec46b5dd02 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:15.747204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52cb30-a53f-40d4-9399-d3ec46b5dd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:15.747714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52cb30-a53f-40d4-9399-d3ec46b5dd02 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:15.748133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52cb30-a53f-40d4-9399-d3ec46b5dd02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:15.749768 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52cb30-a53f-40d4-9399-d3ec46b5dd02 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:15.767819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1165/2333] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:15.772233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d6cde-a773-427a-9c1f-f18df134257e admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:15.782526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d6cde-a773-427a-9c1f-f18df134257e admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:15.787486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f8824-a24b-4550-8df2-45088dec49dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:15.788254 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f8824-a24b-4550-8df2-45088dec49dd admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:15.788470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f8824-a24b-4550-8df2-45088dec49dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:15.788745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f8824-a24b-4550-8df2-45088dec49dd admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:15.789551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f8824-a24b-4550-8df2-45088dec49dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:15.791226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f8824-a24b-4550-8df2-45088dec49dd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:15.803137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1169/2334] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:15.807885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52cb30-a53f-40d4-9399-d3ec46b5dd02 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:15.820199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52cb30-a53f-40d4-9399-d3ec46b5dd02 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:15.824763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f6bc3-6c77-4b09-9607-6a22c622c441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:15.825661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f6bc3-6c77-4b09-9607-6a22c622c441 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:15.825981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f6bc3-6c77-4b09-9607-6a22c622c441 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:15.826260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f6bc3-6c77-4b09-9607-6a22c622c441 admin admin] PATH_INFO: `/v3/projects/d88608ccd5b5447eb6b6804af5520e4b/users/5c8cbf4212404203bc9d998a2a0972e0/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:15.843189 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f6bc3-6c77-4b09-9607-6a22c622c441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88608ccd5b5447eb6b6804af5520e4b, user_id=5c8cbf4212404203bc9d998a2a0972e0, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:15.845091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f6bc3-6c77-4b09-9607-6a22c622c441 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:15.937239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1166/2335] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:15 2022] PUT /identity/v3/projects/d88608ccd5b5447eb6b6804af5520e4b/users/5c8cbf4212404203bc9d998a2a0972e0/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:15.948704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaf773-96a7-4ab8-bd4b-1b38fe614f83 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:15.948996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaf773-96a7-4ab8-bd4b-1b38fe614f83 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:15.949275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbaf773-96a7-4ab8-bd4b-1b38fe614f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:15.979598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7dbaf773-96a7-4ab8-bd4b-1b38fe614f83 None None] MFA Rules not processed for user `5c8cbf4212404203bc9d998a2a0972e0`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:15.980066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7dbaf773-96a7-4ab8-bd4b-1b38fe614f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:16.062507 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1170/2336] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:02:16.069130 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f6bc3-6c77-4b09-9607-6a22c622c441 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:16.137667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f6bc3-6c77-4b09-9607-6a22c622c441 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:16.142570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbaf37-16b3-423e-b341-69eeecaa5bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:16.143248 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbaf37-16b3-423e-b341-69eeecaa5bde admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:16.143535 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbaf37-16b3-423e-b341-69eeecaa5bde admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:16.143811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbaf37-16b3-423e-b341-69eeecaa5bde admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:16.144557 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbaf37-16b3-423e-b341-69eeecaa5bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:16.146778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbaf37-16b3-423e-b341-69eeecaa5bde admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:16.241991 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1167/2337] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:02:16 2022] POST /identity/v3/projects => generated 355 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:16.248934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbaf773-96a7-4ab8-bd4b-1b38fe614f83 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:16.257329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbaf773-96a7-4ab8-bd4b-1b38fe614f83 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:16.269766 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:16.317624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca22623-4e45-49cd-afbf-ccdd273613a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:16.320663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca22623-4e45-49cd-afbf-ccdd273613a7 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:16.320663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca22623-4e45-49cd-afbf-ccdd273613a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:16.320663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca22623-4e45-49cd-afbf-ccdd273613a7 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:16.320663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca22623-4e45-49cd-afbf-ccdd273613a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:16.321530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca22623-4e45-49cd-afbf-ccdd273613a7 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:16.445959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1171/2338] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:02:16 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:16.447429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbaf37-16b3-423e-b341-69eeecaa5bde admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:16.458199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbaf37-16b3-423e-b341-69eeecaa5bde admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:16.462946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe91ce8-7329-4e84-b085-4ca6f7828123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:16.463619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe91ce8-7329-4e84-b085-4ca6f7828123 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:16.464272 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe91ce8-7329-4e84-b085-4ca6f7828123 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:16.464578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe91ce8-7329-4e84-b085-4ca6f7828123 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:16.465316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe91ce8-7329-4e84-b085-4ca6f7828123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:16.467011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe91ce8-7329-4e84-b085-4ca6f7828123 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:16.479336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1168/2339] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:02:16.503409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca22623-4e45-49cd-afbf-ccdd273613a7 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:16.503409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca22623-4e45-49cd-afbf-ccdd273613a7 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:16.503409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8426e7a-fc6f-4f58-98a7-e45338fd098f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:16.503409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8426e7a-fc6f-4f58-98a7-e45338fd098f admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:16.503925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8426e7a-fc6f-4f58-98a7-e45338fd098f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:16.503925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8426e7a-fc6f-4f58-98a7-e45338fd098f admin admin] PATH_INFO: `/v3/projects/2a30822bc3eb4acd85fe824eb6982129/users/7f043dc9e60b4d4496e7ec5ce5fb7030/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:16.521926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8426e7a-fc6f-4f58-98a7-e45338fd098f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a30822bc3eb4acd85fe824eb6982129, user_id=7f043dc9e60b4d4496e7ec5ce5fb7030, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:16.525762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8426e7a-fc6f-4f58-98a7-e45338fd098f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:16.611014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1172/2340] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:16 2022] PUT /identity/v3/projects/2a30822bc3eb4acd85fe824eb6982129/users/7f043dc9e60b4d4496e7ec5ce5fb7030/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:02:16.615369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe91ce8-7329-4e84-b085-4ca6f7828123 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:16.625749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe91ce8-7329-4e84-b085-4ca6f7828123 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:16.665432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:16.698157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9c940-9d45-42d2-97bb-0a0059ce60db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:16.698846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9c940-9d45-42d2-97bb-0a0059ce60db admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:16.699134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9c940-9d45-42d2-97bb-0a0059ce60db admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:16.699415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9c940-9d45-42d2-97bb-0a0059ce60db admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:16.700169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9c940-9d45-42d2-97bb-0a0059ce60db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:16.701788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9c940-9d45-42d2-97bb-0a0059ce60db admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:16.713721 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1169/2341] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:16.717831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8426e7a-fc6f-4f58-98a7-e45338fd098f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:16.729394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8426e7a-fc6f-4f58-98a7-e45338fd098f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:16.736571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652506a-7d16-4d1f-9a57-f2f6d8901085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:16.737475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652506a-7d16-4d1f-9a57-f2f6d8901085 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:16.737758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652506a-7d16-4d1f-9a57-f2f6d8901085 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:16.738089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652506a-7d16-4d1f-9a57-f2f6d8901085 admin admin] PATH_INFO: `/v3/projects/2a30822bc3eb4acd85fe824eb6982129/users/7f043dc9e60b4d4496e7ec5ce5fb7030/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:16.740093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652506a-7d16-4d1f-9a57-f2f6d8901085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a30822bc3eb4acd85fe824eb6982129, user_id=7f043dc9e60b4d4496e7ec5ce5fb7030, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:16.742833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652506a-7d16-4d1f-9a57-f2f6d8901085 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:16.753777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1173/2342] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:16 2022] PUT /identity/v3/projects/2a30822bc3eb4acd85fe824eb6982129/users/7f043dc9e60b4d4496e7ec5ce5fb7030/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:16.764064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd0ca6-d68f-4b07-8888-093afdead963 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:16.764609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd0ca6-d68f-4b07-8888-093afdead963 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:16.765139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd0ca6-d68f-4b07-8888-093afdead963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:16.794838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-dffd0ca6-d68f-4b07-8888-093afdead963 None None] MFA Rules not processed for user `7f043dc9e60b4d4496e7ec5ce5fb7030`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:16.796104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-dffd0ca6-d68f-4b07-8888-093afdead963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:16.866979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1170/2343] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:16 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:16.878290 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652506a-7d16-4d1f-9a57-f2f6d8901085 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:16.880022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7652506a-7d16-4d1f-9a57-f2f6d8901085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:16.945535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652506a-7d16-4d1f-9a57-f2f6d8901085 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:16.949641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74829c-2eeb-4a89-9b32-70ea7f626914 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:16.950425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74829c-2eeb-4a89-9b32-70ea7f626914 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:16.950717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74829c-2eeb-4a89-9b32-70ea7f626914 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:16.950999 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74829c-2eeb-4a89-9b32-70ea7f626914 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:16.961792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74829c-2eeb-4a89-9b32-70ea7f626914 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:16.963389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74829c-2eeb-4a89-9b32-70ea7f626914 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:16.974686 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1174/2344] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:16 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:17.215724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd0ca6-d68f-4b07-8888-093afdead963 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:17.225765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd0ca6-d68f-4b07-8888-093afdead963 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:17.230877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6ea0f-509f-494c-9891-8ee5c81af474 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:17.231547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6ea0f-509f-494c-9891-8ee5c81af474 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:17.231835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6ea0f-509f-494c-9891-8ee5c81af474 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:17.241982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6ea0f-509f-494c-9891-8ee5c81af474 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:17.299304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6ea0f-509f-494c-9891-8ee5c81af474 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:17.300706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6ea0f-509f-494c-9891-8ee5c81af474 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:17.310728 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1171/2345] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:17 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:18.728603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17376bbf-ddf4-40a6-ae1b-5a35154d13ae None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:18.729167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17376bbf-ddf4-40a6-ae1b-5a35154d13ae None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:18.729291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17376bbf-ddf4-40a6-ae1b-5a35154d13ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:18.748998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-17376bbf-ddf4-40a6-ae1b-5a35154d13ae None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:18.750265 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-17376bbf-ddf4-40a6-ae1b-5a35154d13ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:18.811891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1175/2346] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 14 16:02:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:18.824257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6ea0f-509f-494c-9891-8ee5c81af474 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:18.825849 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-36b6ea0f-509f-494c-9891-8ee5c81af474 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:18.883240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:18.907210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6ea0f-509f-494c-9891-8ee5c81af474 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:18.913739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba578ea4-836b-459a-bc7b-3edf664ec152 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:18.914514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba578ea4-836b-459a-bc7b-3edf664ec152 service swift] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:18.914830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba578ea4-836b-459a-bc7b-3edf664ec152 service swift] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:18.915111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba578ea4-836b-459a-bc7b-3edf664ec152 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:18.925589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba578ea4-836b-459a-bc7b-3edf664ec152 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:18.927056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba578ea4-836b-459a-bc7b-3edf664ec152 service swift] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:18.937922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1172/2347] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 16:02:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 16:02:28.804929 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:28.936973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=10 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:37.280295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d86bd6-3018-47bb-91a1-928f3e16c0c5 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:37.280818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d86bd6-3018-47bb-91a1-928f3e16c0c5 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:37.280930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d86bd6-3018-47bb-91a1-928f3e16c0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:37.298691 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-82d86bd6-3018-47bb-91a1-928f3e16c0c5 None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:37.299902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-82d86bd6-3018-47bb-91a1-928f3e16c0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:37.307059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1176/2348] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 14 16:02:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:37.320120 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba578ea4-836b-459a-bc7b-3edf664ec152 service swift] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:37.330829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba578ea4-836b-459a-bc7b-3edf664ec152 service swift] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:37.336470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2a68d-a7d5-4a0e-b2b8-fba35e085304 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:37.337119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2a68d-a7d5-4a0e-b2b8-fba35e085304 service swift] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:37.337414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2a68d-a7d5-4a0e-b2b8-fba35e085304 service swift] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:37.337692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2a68d-a7d5-4a0e-b2b8-fba35e085304 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:37.346154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2a68d-a7d5-4a0e-b2b8-fba35e085304 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:37.347722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2a68d-a7d5-4a0e-b2b8-fba35e085304 service swift] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:37.356053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1173/2349] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 16:02:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 16:02:38.551983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d86bd6-3018-47bb-91a1-928f3e16c0c5 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:38.622279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d86bd6-3018-47bb-91a1-928f3e16c0c5 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:38.626805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:38.627710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:38.627998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:38.628921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] PATH_INFO: `/v3/users/5c8cbf4212404203bc9d998a2a0972e0` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:38.630206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c8cbf4212404203bc9d998a2a0972e0)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:38.632140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:38.820135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c8cbf4212404203bc9d998a2a0972e0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:38.924733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c8cbf4212404203bc9d998a2a0972e0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:38.925248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c8cbf4212404203bc9d998a2a0972e0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:38.933012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c8cbf4212404203bc9d998a2a0972e0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:38.936219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c8cbf4212404203bc9d998a2a0972e0'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:38.953577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1177/2350] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:02:38 2022] DELETE /identity/v3/users/5c8cbf4212404203bc9d998a2a0972e0 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:38.958971 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e567328-3096-4bef-9623-808fcffbc273 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:38.959461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e567328-3096-4bef-9623-808fcffbc273 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:38.959699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e567328-3096-4bef-9623-808fcffbc273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:38.978144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0e567328-3096-4bef-9623-808fcffbc273 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:38.979445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0e567328-3096-4bef-9623-808fcffbc273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:39.045706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1174/2351] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:39.056468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:39.059595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:39.128628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deb874-490b-4cf9-88c0-6a74aaf98337 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:39.128628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011120b-ae61-4416-8e98-ed2d354f841f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:39.128628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011120b-ae61-4416-8e98-ed2d354f841f service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:39.128628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011120b-ae61-4416-8e98-ed2d354f841f service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:39.129247 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011120b-ae61-4416-8e98-ed2d354f841f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:39.147506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011120b-ae61-4416-8e98-ed2d354f841f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:39.148959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011120b-ae61-4416-8e98-ed2d354f841f service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:39.158971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1178/2352] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:39.573083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e567328-3096-4bef-9623-808fcffbc273 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:39.576441 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0e567328-3096-4bef-9623-808fcffbc273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:39.596221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e567328-3096-4bef-9623-808fcffbc273 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:39.613738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:39.616498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:39.616787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:39.617071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] PATH_INFO: `/v3/projects/d88608ccd5b5447eb6b6804af5520e4b` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:39.618949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d88608ccd5b5447eb6b6804af5520e4b)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:39.620904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:39.714796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88608ccd5b5447eb6b6804af5520e4b', 'request_id': 'req-5eede616-16ea-49f9-bf96-6584450550c7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:39.723315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88608ccd5b5447eb6b6804af5520e4b', 'request_id': 'req-5eede616-16ea-49f9-bf96-6584450550c7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:39.768664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88608ccd5b5447eb6b6804af5520e4b', 'request_id': 'req-5eede616-16ea-49f9-bf96-6584450550c7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:39.784360 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] The token cache is being invalidate because project d88608ccd5b5447eb6b6804af5520e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:02:39.784610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:39.785354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:39.787484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1175/2353] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:02:39 2022] DELETE /identity/v3/projects/d88608ccd5b5447eb6b6804af5520e4b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:39.791649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011120b-ae61-4416-8e98-ed2d354f841f service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:39.806420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011120b-ae61-4416-8e98-ed2d354f841f service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:39.823021 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:39.882023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:39.883552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:39.884106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:39.884526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] PATH_INFO: `/v3/users/7f043dc9e60b4d4496e7ec5ce5fb7030` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:39.886769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f043dc9e60b4d4496e7ec5ce5fb7030)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:39.890285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:40.179309 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f043dc9e60b4d4496e7ec5ce5fb7030'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:40.233243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f043dc9e60b4d4496e7ec5ce5fb7030'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:40.233912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f043dc9e60b4d4496e7ec5ce5fb7030'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:40.242157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f043dc9e60b4d4496e7ec5ce5fb7030'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:40.242927 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e23349a-6d6d-4e4b-87fb-f23589caabbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f043dc9e60b4d4496e7ec5ce5fb7030'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:40.269973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1179/2354] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:02:39 2022] DELETE /identity/v3/users/7f043dc9e60b4d4496e7ec5ce5fb7030 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:40.744001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:40.745457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:40.759900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eede616-16ea-49f9-bf96-6584450550c7 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:40.839166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:40.840057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:40.840374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:40.840629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] PATH_INFO: `/v3/projects/2a30822bc3eb4acd85fe824eb6982129` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:40.841898 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a30822bc3eb4acd85fe824eb6982129)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:40.843982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:40.861112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:40.955199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a30822bc3eb4acd85fe824eb6982129', 'request_id': 'req-d9f34bd7-ce38-442f-9e85-5452e3b14765'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:40.961231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a30822bc3eb4acd85fe824eb6982129', 'request_id': 'req-d9f34bd7-ce38-442f-9e85-5452e3b14765'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:41.055973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a30822bc3eb4acd85fe824eb6982129', 'request_id': 'req-d9f34bd7-ce38-442f-9e85-5452e3b14765'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:41.069619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] The token cache is being invalidate because project 2a30822bc3eb4acd85fe824eb6982129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:02:41.069934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:41.070583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d9f34bd7-ce38-442f-9e85-5452e3b14765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:41.072132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1176/2355] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:02:40 2022] DELETE /identity/v3/projects/2a30822bc3eb4acd85fe824eb6982129 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:41.077262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c358c-4770-473b-b5a2-e710b357760d None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:41.077752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c358c-4770-473b-b5a2-e710b357760d None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:41.077888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c358c-4770-473b-b5a2-e710b357760d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:41.096646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-345c358c-4770-473b-b5a2-e710b357760d None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:41.097895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-345c358c-4770-473b-b5a2-e710b357760d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:41.165463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1180/2356] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:41.180886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653210f-edda-4621-a601-c314019565b5 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:41.181627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653210f-edda-4621-a601-c314019565b5 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:41.181945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653210f-edda-4621-a601-c314019565b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:41.201473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-3653210f-edda-4621-a601-c314019565b5 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:41.202824 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3653210f-edda-4621-a601-c314019565b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:41.211498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1177/2357] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:41.215891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c358c-4770-473b-b5a2-e710b357760d None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:41.227225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c358c-4770-473b-b5a2-e710b357760d None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:41.232209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669e726-263c-4b09-a13f-37304c2bbc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:41.232893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669e726-263c-4b09-a13f-37304c2bbc49 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:41.233173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669e726-263c-4b09-a13f-37304c2bbc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:41.233450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669e726-263c-4b09-a13f-37304c2bbc49 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:41.234220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669e726-263c-4b09-a13f-37304c2bbc49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:02:41.234525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669e726-263c-4b09-a13f-37304c2bbc49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:41.236200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669e726-263c-4b09-a13f-37304c2bbc49 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:41.253002 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1181/2358] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:02:41.254526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653210f-edda-4621-a601-c314019565b5 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:41.265341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653210f-edda-4621-a601-c314019565b5 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:41.270139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e213026-3e46-41c5-a5c4-d4fe75d7fc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:41.270806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e213026-3e46-41c5-a5c4-d4fe75d7fc34 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:41.271091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e213026-3e46-41c5-a5c4-d4fe75d7fc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:41.271391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e213026-3e46-41c5-a5c4-d4fe75d7fc34 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:41.277708 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e213026-3e46-41c5-a5c4-d4fe75d7fc34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:41.280494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e213026-3e46-41c5-a5c4-d4fe75d7fc34 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:41.497680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1178/2359] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:02:41 2022] POST /identity/v3/projects => generated 389 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:41.501917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669e726-263c-4b09-a13f-37304c2bbc49 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:41.514127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669e726-263c-4b09-a13f-37304c2bbc49 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:41.534660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:41.596727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2f217-6b43-40f1-9ceb-02b44e2d67bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:41.597856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2f217-6b43-40f1-9ceb-02b44e2d67bd admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:41.597856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2f217-6b43-40f1-9ceb-02b44e2d67bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:41.598079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2f217-6b43-40f1-9ceb-02b44e2d67bd admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:41.599275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2f217-6b43-40f1-9ceb-02b44e2d67bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:41.601479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2f217-6b43-40f1-9ceb-02b44e2d67bd admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:41.731404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1182/2360] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:02:41 2022] POST /identity/v3/users => generated 295 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:41.735910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e213026-3e46-41c5-a5c4-d4fe75d7fc34 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:41.747425 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e213026-3e46-41c5-a5c4-d4fe75d7fc34 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:41.753118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb56e4-13e9-4398-a113-a4572ef84818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:41.754019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb56e4-13e9-4398-a113-a4572ef84818 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:41.754370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb56e4-13e9-4398-a113-a4572ef84818 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:41.755240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb56e4-13e9-4398-a113-a4572ef84818 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:41.756233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb56e4-13e9-4398-a113-a4572ef84818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:41.757994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb56e4-13e9-4398-a113-a4572ef84818 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:41.770340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1179/2361] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:41.775146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2f217-6b43-40f1-9ceb-02b44e2d67bd admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:41.785958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2f217-6b43-40f1-9ceb-02b44e2d67bd admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:41.790958 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2261c-7e38-441d-9bb8-0838ea188c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:41.791640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2261c-7e38-441d-9bb8-0838ea188c95 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:41.791922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2261c-7e38-441d-9bb8-0838ea188c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:41.792198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2261c-7e38-441d-9bb8-0838ea188c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:41.792939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2261c-7e38-441d-9bb8-0838ea188c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:41.794588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2261c-7e38-441d-9bb8-0838ea188c95 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:41.809164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1183/2362] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:41.810438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb56e4-13e9-4398-a113-a4572ef84818 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:41.821231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb56e4-13e9-4398-a113-a4572ef84818 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:41.826071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60720d-eaea-4e8a-b47c-bfa6d1c4f195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:41.826913 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60720d-eaea-4e8a-b47c-bfa6d1c4f195 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:41.827191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60720d-eaea-4e8a-b47c-bfa6d1c4f195 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:41.827470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60720d-eaea-4e8a-b47c-bfa6d1c4f195 admin admin] PATH_INFO: `/v3/projects/5b9f6c1bb40f47cf97ad1248722d5ea7/users/e14fa9718b624541b55c9ea8865d1812/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:41.845195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60720d-eaea-4e8a-b47c-bfa6d1c4f195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b9f6c1bb40f47cf97ad1248722d5ea7, user_id=e14fa9718b624541b55c9ea8865d1812, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:41.847919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60720d-eaea-4e8a-b47c-bfa6d1c4f195 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:42.006524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1180/2363] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:41 2022] PUT /identity/v3/projects/5b9f6c1bb40f47cf97ad1248722d5ea7/users/e14fa9718b624541b55c9ea8865d1812/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:42.017088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8d6a5-1927-4fee-b0c7-f5cb5cd917d4 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:42.017378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8d6a5-1927-4fee-b0c7-f5cb5cd917d4 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:42.017657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8d6a5-1927-4fee-b0c7-f5cb5cd917d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:42.059379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7db8d6a5-1927-4fee-b0c7-f5cb5cd917d4 None None] MFA Rules not processed for user `e14fa9718b624541b55c9ea8865d1812`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:42.060975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7db8d6a5-1927-4fee-b0c7-f5cb5cd917d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:42.163675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1184/2364] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:42.174683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60720d-eaea-4e8a-b47c-bfa6d1c4f195 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:42.257066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60720d-eaea-4e8a-b47c-bfa6d1c4f195 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:42.273249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54940ca-e261-462a-b055-71bd051366a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:42.274010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54940ca-e261-462a-b055-71bd051366a8 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:42.274299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54940ca-e261-462a-b055-71bd051366a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:42.275163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54940ca-e261-462a-b055-71bd051366a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:42.275950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54940ca-e261-462a-b055-71bd051366a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:42.277998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54940ca-e261-462a-b055-71bd051366a8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:42.416608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1181/2365] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:02:42 2022] POST /identity/v3/projects => generated 387 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:42.421013 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8d6a5-1927-4fee-b0c7-f5cb5cd917d4 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:42.435252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8d6a5-1927-4fee-b0c7-f5cb5cd917d4 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:42.530798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fef6b0-4aea-4002-b460-8b92bf896e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:42.531497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fef6b0-4aea-4002-b460-8b92bf896e9c admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:42.531772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fef6b0-4aea-4002-b460-8b92bf896e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:42.532048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fef6b0-4aea-4002-b460-8b92bf896e9c admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:42.532903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fef6b0-4aea-4002-b460-8b92bf896e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:42.535144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fef6b0-4aea-4002-b460-8b92bf896e9c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:42.648415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1185/2366] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:02:42 2022] POST /identity/v3/users => generated 300 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:42.652732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54940ca-e261-462a-b055-71bd051366a8 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:42.676888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54940ca-e261-462a-b055-71bd051366a8 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:42.693391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae60c1-b602-46a4-9108-75cffa4a5abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:42.695696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae60c1-b602-46a4-9108-75cffa4a5abb admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:42.696116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae60c1-b602-46a4-9108-75cffa4a5abb admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:42.696473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae60c1-b602-46a4-9108-75cffa4a5abb admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:42.697438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae60c1-b602-46a4-9108-75cffa4a5abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:42.701392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae60c1-b602-46a4-9108-75cffa4a5abb admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:42.718114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1182/2367] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:02:42.721881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fef6b0-4aea-4002-b460-8b92bf896e9c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:42.764054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fef6b0-4aea-4002-b460-8b92bf896e9c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:42.769532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018da48d-b773-44fa-a1a1-1f959aebc28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:42.771036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018da48d-b773-44fa-a1a1-1f959aebc28a admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:42.771330 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018da48d-b773-44fa-a1a1-1f959aebc28a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:42.771619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018da48d-b773-44fa-a1a1-1f959aebc28a admin admin] PATH_INFO: `/v3/projects/db0d1ea148c249879e272abc78370b34/users/953cb4b848a14053b6bc1d3a0f054eb9/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:42.799988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018da48d-b773-44fa-a1a1-1f959aebc28a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0d1ea148c249879e272abc78370b34, user_id=953cb4b848a14053b6bc1d3a0f054eb9, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:42.805292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018da48d-b773-44fa-a1a1-1f959aebc28a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:42.811388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:42.913074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1186/2368] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:42 2022] PUT /identity/v3/projects/db0d1ea148c249879e272abc78370b34/users/953cb4b848a14053b6bc1d3a0f054eb9/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:42.918169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae60c1-b602-46a4-9108-75cffa4a5abb admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:42.928199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae60c1-b602-46a4-9108-75cffa4a5abb admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:42.941298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:43.015173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8356986-51c1-416c-9062-8d345d278984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:43.016525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8356986-51c1-416c-9062-8d345d278984 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:43.016815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8356986-51c1-416c-9062-8d345d278984 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:43.017103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8356986-51c1-416c-9062-8d345d278984 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:43.018662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8356986-51c1-416c-9062-8d345d278984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:43.020523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8356986-51c1-416c-9062-8d345d278984 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:43.040220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1183/2369] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:43.041415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018da48d-b773-44fa-a1a1-1f959aebc28a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:43.055029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018da48d-b773-44fa-a1a1-1f959aebc28a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:43.061579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f891e-c769-451b-bd7c-e2ab6c617ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:43.062574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f891e-c769-451b-bd7c-e2ab6c617ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:43.063115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f891e-c769-451b-bd7c-e2ab6c617ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:43.063226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f891e-c769-451b-bd7c-e2ab6c617ca5 admin admin] PATH_INFO: `/v3/projects/db0d1ea148c249879e272abc78370b34/users/953cb4b848a14053b6bc1d3a0f054eb9/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:43.066498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f891e-c769-451b-bd7c-e2ab6c617ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0d1ea148c249879e272abc78370b34, user_id=953cb4b848a14053b6bc1d3a0f054eb9, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:43.068885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f891e-c769-451b-bd7c-e2ab6c617ca5 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:43.081933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1187/2370] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:43 2022] PUT /identity/v3/projects/db0d1ea148c249879e272abc78370b34/users/953cb4b848a14053b6bc1d3a0f054eb9/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:43.091942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffdc42-6052-4e27-84d4-464b3cf76ee8 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:43.092240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffdc42-6052-4e27-84d4-464b3cf76ee8 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:43.092523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffdc42-6052-4e27-84d4-464b3cf76ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:43.126843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e9ffdc42-6052-4e27-84d4-464b3cf76ee8 None None] MFA Rules not processed for user `953cb4b848a14053b6bc1d3a0f054eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:43.127990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e9ffdc42-6052-4e27-84d4-464b3cf76ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:43.233246 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1184/2371] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:43.245118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f891e-c769-451b-bd7c-e2ab6c617ca5 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:43.247050 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-941f891e-c769-451b-bd7c-e2ab6c617ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:43.322546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f891e-c769-451b-bd7c-e2ab6c617ca5 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:43.326835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4c4a2-e4ed-467c-a1a0-298931b85fd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:43.328601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4c4a2-e4ed-467c-a1a0-298931b85fd7 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:43.328844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4c4a2-e4ed-467c-a1a0-298931b85fd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:43.329129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4c4a2-e4ed-467c-a1a0-298931b85fd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:43.341391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4c4a2-e4ed-467c-a1a0-298931b85fd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:43.343537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4c4a2-e4ed-467c-a1a0-298931b85fd7 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:43.356144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1188/2372] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:02:43.506337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffdc42-6052-4e27-84d4-464b3cf76ee8 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:43.578277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffdc42-6052-4e27-84d4-464b3cf76ee8 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:43.583216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:43.584111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:43.584401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:43.584689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] PATH_INFO: `/v3/users/e14fa9718b624541b55c9ea8865d1812` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:43.585931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e14fa9718b624541b55c9ea8865d1812)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:43.587873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:43.861199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:43.870769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e14fa9718b624541b55c9ea8865d1812'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:43.936314 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e14fa9718b624541b55c9ea8865d1812'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:43.937024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e14fa9718b624541b55c9ea8865d1812'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:43.945136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e14fa9718b624541b55c9ea8865d1812'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:43.953401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e14fa9718b624541b55c9ea8865d1812'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:43.965322 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1185/2373] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:02:43 2022] DELETE /identity/v3/users/e14fa9718b624541b55c9ea8865d1812 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:43.970148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b0bc3-3227-4106-a4ec-f8deba8b14d3 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:43.970436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b0bc3-3227-4106-a4ec-f8deba8b14d3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:43.970813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b0bc3-3227-4106-a4ec-f8deba8b14d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:43.990393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b59b0bc3-3227-4106-a4ec-f8deba8b14d3 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:43.991995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b59b0bc3-3227-4106-a4ec-f8deba8b14d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:44.057188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1189/2374] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:44.072213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:44.073667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:44.137080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838164d-6d65-4ed8-98e1-95ad37afcbe6 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:44.142570 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee343e5-3a20-493e-9e9d-6937cd1dc448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:44.143164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee343e5-3a20-493e-9e9d-6937cd1dc448 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:44.143457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee343e5-3a20-493e-9e9d-6937cd1dc448 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:44.143743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee343e5-3a20-493e-9e9d-6937cd1dc448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:44.155826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee343e5-3a20-493e-9e9d-6937cd1dc448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:44.157523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee343e5-3a20-493e-9e9d-6937cd1dc448 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:44.169097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1186/2375] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:44.392033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be1b86-4c5a-49db-a89f-ac52f81a323e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:44.392552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be1b86-4c5a-49db-a89f-ac52f81a323e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:44.400891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be1b86-4c5a-49db-a89f-ac52f81a323e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:44.410687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-e0be1b86-4c5a-49db-a89f-ac52f81a323e None None] MFA Rules not processed for user `ba765c1a8c6c449ca699eb7a44dde50b`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:44.411950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e0be1b86-4c5a-49db-a89f-ac52f81a323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:44.470612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1190/2376] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:44.651021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee343e5-3a20-493e-9e9d-6937cd1dc448 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:44.652951 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2ee343e5-3a20-493e-9e9d-6937cd1dc448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:44.678788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee343e5-3a20-493e-9e9d-6937cd1dc448 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:44.685191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:44.686311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:44.686673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:44.687014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] PATH_INFO: `/v3/projects/5b9f6c1bb40f47cf97ad1248722d5ea7` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:44.688283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b9f6c1bb40f47cf97ad1248722d5ea7)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:44.690345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:44.869519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b9f6c1bb40f47cf97ad1248722d5ea7', 'request_id': 'req-28744b04-12a8-497a-bc76-07b5cc21cc8b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:44.880469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b9f6c1bb40f47cf97ad1248722d5ea7', 'request_id': 'req-28744b04-12a8-497a-bc76-07b5cc21cc8b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:44.959001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b9f6c1bb40f47cf97ad1248722d5ea7', 'request_id': 'req-28744b04-12a8-497a-bc76-07b5cc21cc8b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:44.974186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] The token cache is being invalidate because project 5b9f6c1bb40f47cf97ad1248722d5ea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:02:44.974502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:44.975617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:44.977554 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1187/2377] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:02:44 2022] DELETE /identity/v3/projects/5b9f6c1bb40f47cf97ad1248722d5ea7 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:02:44.982716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be1b86-4c5a-49db-a89f-ac52f81a323e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:45.003459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be1b86-4c5a-49db-a89f-ac52f81a323e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:45.035213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:45.068548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:45.069506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:45.069894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:45.070085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] PATH_INFO: `/v3/users/953cb4b848a14053b6bc1d3a0f054eb9` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:45.071360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953cb4b848a14053b6bc1d3a0f054eb9)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:45.073518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:45.320187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953cb4b848a14053b6bc1d3a0f054eb9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:45.392807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953cb4b848a14053b6bc1d3a0f054eb9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:45.393475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953cb4b848a14053b6bc1d3a0f054eb9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:45.402459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953cb4b848a14053b6bc1d3a0f054eb9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:45.403142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-2b0f5932-6140-491d-881d-a1ad9a6471e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953cb4b848a14053b6bc1d3a0f054eb9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:45.421777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1191/2378] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:02:44 2022] DELETE /identity/v3/users/953cb4b848a14053b6bc1d3a0f054eb9 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:02:45.972590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:45.974939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:45.989343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28744b04-12a8-497a-bc76-07b5cc21cc8b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:46.040813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:46.081748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:46.083246 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:46.083532 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:46.083810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] PATH_INFO: `/v3/projects/db0d1ea148c249879e272abc78370b34` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:46.085032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db0d1ea148c249879e272abc78370b34)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:46.087059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:46.330299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db0d1ea148c249879e272abc78370b34', 'request_id': 'req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:46.337715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db0d1ea148c249879e272abc78370b34', 'request_id': 'req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:46.501661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db0d1ea148c249879e272abc78370b34', 'request_id': 'req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:46.514806 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] The token cache is being invalidate because project db0d1ea148c249879e272abc78370b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:02:46.515097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:46.515771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-106c7e9c-7cb9-4b89-93ae-a4ce9b08375e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:02:46.517552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1188/2379] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:02:45 2022] DELETE /identity/v3/projects/db0d1ea148c249879e272abc78370b34 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:46.523314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804065ea-14ec-4d25-bc34-73741681eea3 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:46.524384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804065ea-14ec-4d25-bc34-73741681eea3 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:46.524384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804065ea-14ec-4d25-bc34-73741681eea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:46.542778 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-804065ea-14ec-4d25-bc34-73741681eea3 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:46.544007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-804065ea-14ec-4d25-bc34-73741681eea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:46.607336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1192/2380] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:46.616684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441166e-26d0-4b57-b00b-87e86638ffaf None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:46.617190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441166e-26d0-4b57-b00b-87e86638ffaf None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:46.617649 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441166e-26d0-4b57-b00b-87e86638ffaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:46.642456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-2441166e-26d0-4b57-b00b-87e86638ffaf None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:46.643690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2441166e-26d0-4b57-b00b-87e86638ffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:46.651921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1189/2381] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:46.656342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804065ea-14ec-4d25-bc34-73741681eea3 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:46.666438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804065ea-14ec-4d25-bc34-73741681eea3 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:46.671220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5835ea-fc0c-4c2b-b6ee-eacde21e0fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:46.671905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5835ea-fc0c-4c2b-b6ee-eacde21e0fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:46.672444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5835ea-fc0c-4c2b-b6ee-eacde21e0fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:46.672791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5835ea-fc0c-4c2b-b6ee-eacde21e0fe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:46.673560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5835ea-fc0c-4c2b-b6ee-eacde21e0fe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:02:46.673904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5835ea-fc0c-4c2b-b6ee-eacde21e0fe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:46.675526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5835ea-fc0c-4c2b-b6ee-eacde21e0fe6 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:46.688568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1193/2382] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:02:46.693988 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441166e-26d0-4b57-b00b-87e86638ffaf None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:46.703671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441166e-26d0-4b57-b00b-87e86638ffaf None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:46.708698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea355a7f-f95e-4081-9109-6d4f99ecb053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:46.709387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea355a7f-f95e-4081-9109-6d4f99ecb053 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:46.709712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea355a7f-f95e-4081-9109-6d4f99ecb053 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:46.710068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea355a7f-f95e-4081-9109-6d4f99ecb053 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:46.710848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea355a7f-f95e-4081-9109-6d4f99ecb053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:46.712813 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea355a7f-f95e-4081-9109-6d4f99ecb053 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:46.791555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1190/2383] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:02:46 2022] POST /identity/v3/projects => generated 409 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:46.793316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5835ea-fc0c-4c2b-b6ee-eacde21e0fe6 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:46.804679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5835ea-fc0c-4c2b-b6ee-eacde21e0fe6 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:46.846509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:46.870777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91074b30-f1d0-46ff-8985-3d033bc43495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:46.871522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91074b30-f1d0-46ff-8985-3d033bc43495 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:46.871915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91074b30-f1d0-46ff-8985-3d033bc43495 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:46.872287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91074b30-f1d0-46ff-8985-3d033bc43495 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:46.873103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91074b30-f1d0-46ff-8985-3d033bc43495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:46.875287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91074b30-f1d0-46ff-8985-3d033bc43495 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:47.068216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1194/2384] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:02:46 2022] POST /identity/v3/users => generated 305 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:47.072522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea355a7f-f95e-4081-9109-6d4f99ecb053 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:47.083630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea355a7f-f95e-4081-9109-6d4f99ecb053 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:47.088311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10232a6-9425-44aa-a141-5cee7007b723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:47.089000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10232a6-9425-44aa-a141-5cee7007b723 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:47.089285 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10232a6-9425-44aa-a141-5cee7007b723 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:47.089561 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10232a6-9425-44aa-a141-5cee7007b723 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:47.090336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10232a6-9425-44aa-a141-5cee7007b723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:47.093110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10232a6-9425-44aa-a141-5cee7007b723 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:47.104734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1191/2385] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:47.109039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91074b30-f1d0-46ff-8985-3d033bc43495 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:47.120662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91074b30-f1d0-46ff-8985-3d033bc43495 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:47.125293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8121f6-eb4b-4f95-ba24-18fd75682a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:47.125936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8121f6-eb4b-4f95-ba24-18fd75682a08 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:47.126220 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8121f6-eb4b-4f95-ba24-18fd75682a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:47.126493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8121f6-eb4b-4f95-ba24-18fd75682a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:47.127215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8121f6-eb4b-4f95-ba24-18fd75682a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:47.129034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8121f6-eb4b-4f95-ba24-18fd75682a08 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:47.140212 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1195/2386] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:47.144777 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10232a6-9425-44aa-a141-5cee7007b723 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:47.154970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10232a6-9425-44aa-a141-5cee7007b723 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:47.160032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dd604-445f-4a44-ba4a-5608c2143e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:47.161432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd604-445f-4a44-ba4a-5608c2143e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:47.161714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd604-445f-4a44-ba4a-5608c2143e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:47.162020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dd604-445f-4a44-ba4a-5608c2143e48 admin admin] PATH_INFO: `/v3/projects/5bea7b3cbfbe4527a427ddd8d976b11b/users/14704f6bb767446eb40d2f608e332462/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:47.177951 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dd604-445f-4a44-ba4a-5608c2143e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bea7b3cbfbe4527a427ddd8d976b11b, user_id=14704f6bb767446eb40d2f608e332462, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:47.180659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dd604-445f-4a44-ba4a-5608c2143e48 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:47.245409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1192/2387] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:47 2022] PUT /identity/v3/projects/5bea7b3cbfbe4527a427ddd8d976b11b/users/14704f6bb767446eb40d2f608e332462/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:02:47.256875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275d365-9bb1-479b-9e8f-55e0014d0990 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:47.257168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275d365-9bb1-479b-9e8f-55e0014d0990 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:47.257513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275d365-9bb1-479b-9e8f-55e0014d0990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:47.293057 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-9275d365-9bb1-479b-9e8f-55e0014d0990 None None] MFA Rules not processed for user `14704f6bb767446eb40d2f608e332462`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:47.295384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9275d365-9bb1-479b-9e8f-55e0014d0990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:47.394243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1196/2388] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:47 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:47.399112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dd604-445f-4a44-ba4a-5608c2143e48 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:47.468568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dd604-445f-4a44-ba4a-5608c2143e48 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:47.474054 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5259b3-160a-4543-a9d2-452fa832905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:47.474729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5259b3-160a-4543-a9d2-452fa832905a admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:47.475018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5259b3-160a-4543-a9d2-452fa832905a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:47.475297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5259b3-160a-4543-a9d2-452fa832905a admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:47.476182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5259b3-160a-4543-a9d2-452fa832905a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:47.478250 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5259b3-160a-4543-a9d2-452fa832905a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:47.487716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:47.566131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1193/2389] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:02:47 2022] POST /identity/v3/projects => generated 409 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:02:47.570523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275d365-9bb1-479b-9e8f-55e0014d0990 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:47.582210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275d365-9bb1-479b-9e8f-55e0014d0990 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:47.642402 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20197daf-837a-4249-8958-452451cb2982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:47.642978 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20197daf-837a-4249-8958-452451cb2982 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:47.643263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20197daf-837a-4249-8958-452451cb2982 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:47.643542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20197daf-837a-4249-8958-452451cb2982 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:47.644252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20197daf-837a-4249-8958-452451cb2982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:47.646732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20197daf-837a-4249-8958-452451cb2982 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:47.799603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1197/2390] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:02:47 2022] POST /identity/v3/users => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:02:47.803765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5259b3-160a-4543-a9d2-452fa832905a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:47.814512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5259b3-160a-4543-a9d2-452fa832905a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:47.819640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791616e-51e5-46d0-bb3e-7f2f5a0db842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:47.820314 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791616e-51e5-46d0-bb3e-7f2f5a0db842 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:47.820601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791616e-51e5-46d0-bb3e-7f2f5a0db842 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:47.820878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791616e-51e5-46d0-bb3e-7f2f5a0db842 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:47.821605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791616e-51e5-46d0-bb3e-7f2f5a0db842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:47.823275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791616e-51e5-46d0-bb3e-7f2f5a0db842 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:47.839689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1194/2391] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:47.842669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20197daf-837a-4249-8958-452451cb2982 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:47.853874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20197daf-837a-4249-8958-452451cb2982 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:47.858846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eea224-220c-498e-81db-9cebe62c0253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:47.859787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eea224-220c-498e-81db-9cebe62c0253 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:47.860071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eea224-220c-498e-81db-9cebe62c0253 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:47.860404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eea224-220c-498e-81db-9cebe62c0253 admin admin] PATH_INFO: `/v3/projects/a87bedb06f144628a28c7d8adaeb5bfc/users/09e6d5bcbdc74e5ebfee8cad15eee672/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:47.876315 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eea224-220c-498e-81db-9cebe62c0253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87bedb06f144628a28c7d8adaeb5bfc, user_id=09e6d5bcbdc74e5ebfee8cad15eee672, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:47.879010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eea224-220c-498e-81db-9cebe62c0253 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:47.976779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1198/2392] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:47 2022] PUT /identity/v3/projects/a87bedb06f144628a28c7d8adaeb5bfc/users/09e6d5bcbdc74e5ebfee8cad15eee672/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:47.980858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791616e-51e5-46d0-bb3e-7f2f5a0db842 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:47.991294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791616e-51e5-46d0-bb3e-7f2f5a0db842 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:48.058487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe05e6-cd47-4b50-aad5-685cd5556b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:48.059181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe05e6-cd47-4b50-aad5-685cd5556b6b admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:48.059528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe05e6-cd47-4b50-aad5-685cd5556b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:48.059742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe05e6-cd47-4b50-aad5-685cd5556b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:48.060536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe05e6-cd47-4b50-aad5-685cd5556b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:48.062253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe05e6-cd47-4b50-aad5-685cd5556b6b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:48.074947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1195/2393] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:02:48.079394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eea224-220c-498e-81db-9cebe62c0253 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:48.090911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eea224-220c-498e-81db-9cebe62c0253 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:48.096073 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c0b0a-0a46-4faa-8f6f-170b4b990e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:48.096994 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c0b0a-0a46-4faa-8f6f-170b4b990e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:48.097305 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c0b0a-0a46-4faa-8f6f-170b4b990e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:48.097599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c0b0a-0a46-4faa-8f6f-170b4b990e78 admin admin] PATH_INFO: `/v3/projects/a87bedb06f144628a28c7d8adaeb5bfc/users/09e6d5bcbdc74e5ebfee8cad15eee672/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:48.099709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c0b0a-0a46-4faa-8f6f-170b4b990e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a87bedb06f144628a28c7d8adaeb5bfc, user_id=09e6d5bcbdc74e5ebfee8cad15eee672, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:48.102463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c0b0a-0a46-4faa-8f6f-170b4b990e78 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:48.108560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:48.114940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1199/2394] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:02:48 2022] PUT /identity/v3/projects/a87bedb06f144628a28c7d8adaeb5bfc/users/09e6d5bcbdc74e5ebfee8cad15eee672/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:02:48.132459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca62184-341c-4036-b8d0-956758a28f01 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:48.132869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca62184-341c-4036-b8d0-956758a28f01 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:48.133212 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca62184-341c-4036-b8d0-956758a28f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:48.162808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-9ca62184-341c-4036-b8d0-956758a28f01 None None] MFA Rules not processed for user `09e6d5bcbdc74e5ebfee8cad15eee672`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:02:48.164029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9ca62184-341c-4036-b8d0-956758a28f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:48.237883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1196/2395] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:02:48 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:02:48.249487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c0b0a-0a46-4faa-8f6f-170b4b990e78 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:48.251382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-254c0b0a-0a46-4faa-8f6f-170b4b990e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:02:48.311986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c0b0a-0a46-4faa-8f6f-170b4b990e78 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:48.315747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab5809-724f-40aa-819c-d9247ce69151 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:48.316440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab5809-724f-40aa-819c-d9247ce69151 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:48.316729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab5809-724f-40aa-819c-d9247ce69151 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:48.317008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab5809-724f-40aa-819c-d9247ce69151 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:48.327485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab5809-724f-40aa-819c-d9247ce69151 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:48.328924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab5809-724f-40aa-819c-d9247ce69151 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:48.339020 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1200/2396] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:48 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:48.765261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca62184-341c-4036-b8d0-956758a28f01 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:48.776197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca62184-341c-4036-b8d0-956758a28f01 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:48.780162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5100b2-1dd3-4c59-934e-bc26dd33e334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:48.781128 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5100b2-1dd3-4c59-934e-bc26dd33e334 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:48.781418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5100b2-1dd3-4c59-934e-bc26dd33e334 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:48.781691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5100b2-1dd3-4c59-934e-bc26dd33e334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:48.855040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5100b2-1dd3-4c59-934e-bc26dd33e334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:48.856501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5100b2-1dd3-4c59-934e-bc26dd33e334 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:02:48.860722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:48.869641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1197/2397] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:02:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:02:58.334512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=27 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:59.527340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab5809-724f-40aa-819c-d9247ce69151 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:02:59.546980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:02:59.638116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab5809-724f-40aa-819c-d9247ce69151 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:02:59.642970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:02:59.643854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:02:59.644177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:02:59.644471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] PATH_INFO: `/v3/users/14704f6bb767446eb40d2f608e332462` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:02:59.645725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14704f6bb767446eb40d2f608e332462)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:02:59.647730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:00.126572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14704f6bb767446eb40d2f608e332462'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:00.175495 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14704f6bb767446eb40d2f608e332462'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:00.176131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14704f6bb767446eb40d2f608e332462'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:00.184023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14704f6bb767446eb40d2f608e332462'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:00.187359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14704f6bb767446eb40d2f608e332462'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:00.204338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1201/2398] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:02:59 2022] DELETE /identity/v3/users/14704f6bb767446eb40d2f608e332462 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:00.209081 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002130-4afb-4ec5-a29d-1a99ad6ba53b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:00.209382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002130-4afb-4ec5-a29d-1a99ad6ba53b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:00.209648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002130-4afb-4ec5-a29d-1a99ad6ba53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:00.266979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-69002130-4afb-4ec5-a29d-1a99ad6ba53b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:00.268863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-69002130-4afb-4ec5-a29d-1a99ad6ba53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:00.330767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1198/2399] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:00.341549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:00.343074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:00.386103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:00.419955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a1d0c-2a92-4903-b0a8-3d08d8e36c91 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:00.424129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0428dd-9f4c-4030-aec3-3f550a07937a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:00.427986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0428dd-9f4c-4030-aec3-3f550a07937a service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:00.427986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0428dd-9f4c-4030-aec3-3f550a07937a service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:00.427986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0428dd-9f4c-4030-aec3-3f550a07937a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:00.435486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0428dd-9f4c-4030-aec3-3f550a07937a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:00.436977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0428dd-9f4c-4030-aec3-3f550a07937a service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:00.449262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1202/2400] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:00.897987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69002130-4afb-4ec5-a29d-1a99ad6ba53b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:00.899445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-69002130-4afb-4ec5-a29d-1a99ad6ba53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:00.911684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69002130-4afb-4ec5-a29d-1a99ad6ba53b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:00.918989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:00.918989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:00.918989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:00.918989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] PATH_INFO: `/v3/projects/5bea7b3cbfbe4527a427ddd8d976b11b` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:00.924620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bea7b3cbfbe4527a427ddd8d976b11b)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:00.926651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:01.098177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bea7b3cbfbe4527a427ddd8d976b11b', 'request_id': 'req-e489100d-0cc3-424a-80fb-ddb8c1ef7598'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.105387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bea7b3cbfbe4527a427ddd8d976b11b', 'request_id': 'req-e489100d-0cc3-424a-80fb-ddb8c1ef7598'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.199948 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bea7b3cbfbe4527a427ddd8d976b11b', 'request_id': 'req-e489100d-0cc3-424a-80fb-ddb8c1ef7598'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.212924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] The token cache is being invalidate because project 5bea7b3cbfbe4527a427ddd8d976b11b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:01.213238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.213941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.215530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1199/2401] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:00 2022] DELETE /identity/v3/projects/5bea7b3cbfbe4527a427ddd8d976b11b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:01.219790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0428dd-9f4c-4030-aec3-3f550a07937a service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:01.230403 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0428dd-9f4c-4030-aec3-3f550a07937a service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:01.289783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:01.290708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:01.290992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:01.291767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] PATH_INFO: `/v3/users/09e6d5bcbdc74e5ebfee8cad15eee672` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:01.292983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e6d5bcbdc74e5ebfee8cad15eee672)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:01.294948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:01.501226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e6d5bcbdc74e5ebfee8cad15eee672'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.586792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e6d5bcbdc74e5ebfee8cad15eee672'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.587431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e6d5bcbdc74e5ebfee8cad15eee672'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.595591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e6d5bcbdc74e5ebfee8cad15eee672'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.596247 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-aef90f8d-6fe7-4050-b95d-7239daccf481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e6d5bcbdc74e5ebfee8cad15eee672'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:01.614666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1203/2402] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:01 2022] DELETE /identity/v3/users/09e6d5bcbdc74e5ebfee8cad15eee672 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:02.063007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:02.064496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:02.080004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489100d-0cc3-424a-80fb-ddb8c1ef7598 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:02.137194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:02.143786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:02.144663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:02.144946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:02.145249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] PATH_INFO: `/v3/projects/a87bedb06f144628a28c7d8adaeb5bfc` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:02.146475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a87bedb06f144628a28c7d8adaeb5bfc)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:02.151530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:02.281202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a87bedb06f144628a28c7d8adaeb5bfc', 'request_id': 'req-9e731dc2-9901-4cca-9488-027e45f2a3ea'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:02.287909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a87bedb06f144628a28c7d8adaeb5bfc', 'request_id': 'req-9e731dc2-9901-4cca-9488-027e45f2a3ea'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:02.376221 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a87bedb06f144628a28c7d8adaeb5bfc', 'request_id': 'req-9e731dc2-9901-4cca-9488-027e45f2a3ea'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:02.392377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] The token cache is being invalidate because project a87bedb06f144628a28c7d8adaeb5bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:02.392672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:02.393333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e731dc2-9901-4cca-9488-027e45f2a3ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:02.394935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1200/2403] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:02 2022] DELETE /identity/v3/projects/a87bedb06f144628a28c7d8adaeb5bfc => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:02.400024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed53a8-2594-4f03-8309-3531f6ba235e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:02.400554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed53a8-2594-4f03-8309-3531f6ba235e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:02.400673 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed53a8-2594-4f03-8309-3531f6ba235e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:02.449533 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-2fed53a8-2594-4f03-8309-3531f6ba235e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:02.450688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2fed53a8-2594-4f03-8309-3531f6ba235e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:02.465534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:02.515670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1204/2404] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:02.523792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7b4d-a524-45b0-b234-37707a946736 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:02.524079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7b4d-a524-45b0-b234-37707a946736 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:02.524354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7b4d-a524-45b0-b234-37707a946736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:02.542793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-807d7b4d-a524-45b0-b234-37707a946736 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:02.544038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-807d7b4d-a524-45b0-b234-37707a946736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:02.552692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1201/2405] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:02.557603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed53a8-2594-4f03-8309-3531f6ba235e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:02.567892 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed53a8-2594-4f03-8309-3531f6ba235e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:02.574454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995361fa-455b-4aa5-9258-11aa056d4cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:02.575164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995361fa-455b-4aa5-9258-11aa056d4cca admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:02.575446 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995361fa-455b-4aa5-9258-11aa056d4cca admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:02.575732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995361fa-455b-4aa5-9258-11aa056d4cca admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:02.576481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995361fa-455b-4aa5-9258-11aa056d4cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:02.576784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995361fa-455b-4aa5-9258-11aa056d4cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:02.578738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995361fa-455b-4aa5-9258-11aa056d4cca admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:02.592792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1205/2406] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:03:02.597316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d7b4d-a524-45b0-b234-37707a946736 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:02.611238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d7b4d-a524-45b0-b234-37707a946736 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:02.616319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5017c-39cf-4ad6-ba04-6fcf6d04e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:02.617000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5017c-39cf-4ad6-ba04-6fcf6d04e348 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:02.617284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5017c-39cf-4ad6-ba04-6fcf6d04e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:02.617553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5017c-39cf-4ad6-ba04-6fcf6d04e348 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:02.618299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5017c-39cf-4ad6-ba04-6fcf6d04e348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:02.620872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5017c-39cf-4ad6-ba04-6fcf6d04e348 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:02.709136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1202/2407] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:02 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:02.714012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995361fa-455b-4aa5-9258-11aa056d4cca admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:02.724102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995361fa-455b-4aa5-9258-11aa056d4cca admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:02.787534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec40f3f-d4dd-406a-b9c5-fc13b30caf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:02.788246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec40f3f-d4dd-406a-b9c5-fc13b30caf88 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:02.788529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec40f3f-d4dd-406a-b9c5-fc13b30caf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:02.788807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec40f3f-d4dd-406a-b9c5-fc13b30caf88 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:02.789519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec40f3f-d4dd-406a-b9c5-fc13b30caf88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:02.791516 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec40f3f-d4dd-406a-b9c5-fc13b30caf88 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:02.896244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1206/2408] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:02 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:02.897413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5017c-39cf-4ad6-ba04-6fcf6d04e348 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:02.909139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5017c-39cf-4ad6-ba04-6fcf6d04e348 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:02.914575 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d963b6-9d35-4efb-b951-32c44bbfe8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:02.915268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d963b6-9d35-4efb-b951-32c44bbfe8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:02.915551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d963b6-9d35-4efb-b951-32c44bbfe8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:02.915848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d963b6-9d35-4efb-b951-32c44bbfe8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:02.916578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d963b6-9d35-4efb-b951-32c44bbfe8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:02.918225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d963b6-9d35-4efb-b951-32c44bbfe8d0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:02.929705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1203/2409] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:02.933930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec40f3f-d4dd-406a-b9c5-fc13b30caf88 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:02.944375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec40f3f-d4dd-406a-b9c5-fc13b30caf88 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:02.949724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158a69b-3d79-4231-9655-ccd2bf9fc17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:02.950436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158a69b-3d79-4231-9655-ccd2bf9fc17f admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:02.950719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158a69b-3d79-4231-9655-ccd2bf9fc17f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:02.950995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158a69b-3d79-4231-9655-ccd2bf9fc17f admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:02.951727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158a69b-3d79-4231-9655-ccd2bf9fc17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:02.953338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158a69b-3d79-4231-9655-ccd2bf9fc17f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:02.972657 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1207/2410] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:02.973926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d963b6-9d35-4efb-b951-32c44bbfe8d0 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:02.984511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d963b6-9d35-4efb-b951-32c44bbfe8d0 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:02.993359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d69dad-f86c-4c21-8bef-30fb6e31c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:02.995006 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d69dad-f86c-4c21-8bef-30fb6e31c00c admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:02.995169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d69dad-f86c-4c21-8bef-30fb6e31c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:02.995393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d69dad-f86c-4c21-8bef-30fb6e31c00c admin admin] PATH_INFO: `/v3/projects/28b50e8310394184a07533a5d49cc2c9/users/d41d5c772dd74e38bef091682f399b20/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:03.013946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d69dad-f86c-4c21-8bef-30fb6e31c00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b50e8310394184a07533a5d49cc2c9, user_id=d41d5c772dd74e38bef091682f399b20, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:03.016957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d69dad-f86c-4c21-8bef-30fb6e31c00c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:03.104729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1204/2411] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:02 2022] PUT /identity/v3/projects/28b50e8310394184a07533a5d49cc2c9/users/d41d5c772dd74e38bef091682f399b20/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:03.115776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc501e9-2ba3-4f88-b976-5c7d20438a7a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:03.116307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc501e9-2ba3-4f88-b976-5c7d20438a7a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:03.116683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc501e9-2ba3-4f88-b976-5c7d20438a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:03.145917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-abc501e9-2ba3-4f88-b976-5c7d20438a7a None None] MFA Rules not processed for user `d41d5c772dd74e38bef091682f399b20`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:03.147270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-abc501e9-2ba3-4f88-b976-5c7d20438a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:03.219318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1208/2412] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:03.224089 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d69dad-f86c-4c21-8bef-30fb6e31c00c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:03.293946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d69dad-f86c-4c21-8bef-30fb6e31c00c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:03.299074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b474a4c-d086-45cd-bac8-50beee35a837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:03.299774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b474a4c-d086-45cd-bac8-50beee35a837 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:03.300058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b474a4c-d086-45cd-bac8-50beee35a837 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:03.300337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b474a4c-d086-45cd-bac8-50beee35a837 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:03.301074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b474a4c-d086-45cd-bac8-50beee35a837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:03.303166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b474a4c-d086-45cd-bac8-50beee35a837 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:03.438409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1205/2413] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:03 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:03.440114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc501e9-2ba3-4f88-b976-5c7d20438a7a None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:03.453865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc501e9-2ba3-4f88-b976-5c7d20438a7a None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:03.472971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:03.531670 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acf710-3167-47b5-b9fd-0cc50658105e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:03.533157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acf710-3167-47b5-b9fd-0cc50658105e admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:03.533157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acf710-3167-47b5-b9fd-0cc50658105e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:03.533649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acf710-3167-47b5-b9fd-0cc50658105e admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:03.534629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acf710-3167-47b5-b9fd-0cc50658105e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:03.537790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26acf710-3167-47b5-b9fd-0cc50658105e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:03.620847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1209/2414] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:03 2022] POST /identity/v3/users => generated 299 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:03.624885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b474a4c-d086-45cd-bac8-50beee35a837 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:03.637101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b474a4c-d086-45cd-bac8-50beee35a837 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:03.642464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff199d2-dc45-4e1f-bc67-a6e9970d90f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:03.643203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff199d2-dc45-4e1f-bc67-a6e9970d90f8 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:03.643511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff199d2-dc45-4e1f-bc67-a6e9970d90f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:03.643798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff199d2-dc45-4e1f-bc67-a6e9970d90f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:03.644536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff199d2-dc45-4e1f-bc67-a6e9970d90f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:03.646178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff199d2-dc45-4e1f-bc67-a6e9970d90f8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:03.648583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:03.658549 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1206/2415] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:03.662855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acf710-3167-47b5-b9fd-0cc50658105e admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:03.673477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26acf710-3167-47b5-b9fd-0cc50658105e admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:03.678629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5544f-88c8-469e-a1ea-e159348b6bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:03.679515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5544f-88c8-469e-a1ea-e159348b6bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:03.679852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5544f-88c8-469e-a1ea-e159348b6bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:03.680128 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5544f-88c8-469e-a1ea-e159348b6bfe admin admin] PATH_INFO: `/v3/projects/ba905b9930ff4b9ea10f400b2b70111a/users/27e7540e7b844fecad728d20f5266228/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:03.695531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5544f-88c8-469e-a1ea-e159348b6bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba905b9930ff4b9ea10f400b2b70111a, user_id=27e7540e7b844fecad728d20f5266228, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:03.698725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5544f-88c8-469e-a1ea-e159348b6bfe admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:03.782354 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1210/2416] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:03 2022] PUT /identity/v3/projects/ba905b9930ff4b9ea10f400b2b70111a/users/27e7540e7b844fecad728d20f5266228/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:03.786407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff199d2-dc45-4e1f-bc67-a6e9970d90f8 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:03.796344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff199d2-dc45-4e1f-bc67-a6e9970d90f8 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:03.854471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a57e7-eea2-471a-8398-9f9134112ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:03.855204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a57e7-eea2-471a-8398-9f9134112ebd admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:03.855395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a57e7-eea2-471a-8398-9f9134112ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:03.855671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a57e7-eea2-471a-8398-9f9134112ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:03.856396 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a57e7-eea2-471a-8398-9f9134112ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:03.858025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a57e7-eea2-471a-8398-9f9134112ebd admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:03.868551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1207/2417] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:03.874582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5544f-88c8-469e-a1ea-e159348b6bfe admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:03.886521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5544f-88c8-469e-a1ea-e159348b6bfe admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:03.891355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b77f38-1cbd-48be-ad83-e8ca99fe54a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:03.892241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b77f38-1cbd-48be-ad83-e8ca99fe54a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:03.892518 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b77f38-1cbd-48be-ad83-e8ca99fe54a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:03.892801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b77f38-1cbd-48be-ad83-e8ca99fe54a2 admin admin] PATH_INFO: `/v3/projects/ba905b9930ff4b9ea10f400b2b70111a/users/27e7540e7b844fecad728d20f5266228/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:03.894918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b77f38-1cbd-48be-ad83-e8ca99fe54a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba905b9930ff4b9ea10f400b2b70111a, user_id=27e7540e7b844fecad728d20f5266228, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:03.897539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b77f38-1cbd-48be-ad83-e8ca99fe54a2 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:03.910208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1211/2418] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:03 2022] PUT /identity/v3/projects/ba905b9930ff4b9ea10f400b2b70111a/users/27e7540e7b844fecad728d20f5266228/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:03.980918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de373ea-325d-49d4-a4aa-22694d613536 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:03.981206 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de373ea-325d-49d4-a4aa-22694d613536 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:03.981486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de373ea-325d-49d4-a4aa-22694d613536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:04.007700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-1de373ea-325d-49d4-a4aa-22694d613536 None None] MFA Rules not processed for user `27e7540e7b844fecad728d20f5266228`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:04.009722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1de373ea-325d-49d4-a4aa-22694d613536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:04.086506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1208/2419] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:04.098173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b77f38-1cbd-48be-ad83-e8ca99fe54a2 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:04.099928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-41b77f38-1cbd-48be-ad83-e8ca99fe54a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:04.158618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b77f38-1cbd-48be-ad83-e8ca99fe54a2 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:04.164385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099d452-dec8-47b9-a5f2-d3265a643ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:04.164385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099d452-dec8-47b9-a5f2-d3265a643ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:04.164385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099d452-dec8-47b9-a5f2-d3265a643ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:04.164385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099d452-dec8-47b9-a5f2-d3265a643ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:04.223382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099d452-dec8-47b9-a5f2-d3265a643ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:04.224816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099d452-dec8-47b9-a5f2-d3265a643ee8 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:04.234477 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1212/2420] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:09.676313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de373ea-325d-49d4-a4aa-22694d613536 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:09.677795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1de373ea-325d-49d4-a4aa-22694d613536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:09.690763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de373ea-325d-49d4-a4aa-22694d613536 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:09.694635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a019e7-3f0b-4f65-9c94-4b4c79587821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:09.695304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a019e7-3f0b-4f65-9c94-4b4c79587821 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:09.695594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a019e7-3f0b-4f65-9c94-4b4c79587821 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:09.695877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a019e7-3f0b-4f65-9c94-4b4c79587821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:09.706482 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a019e7-3f0b-4f65-9c94-4b4c79587821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:09.707966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a019e7-3f0b-4f65-9c94-4b4c79587821 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:09.720299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1209/2421] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:09 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:14.230240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=85,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:15.032793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099d452-dec8-47b9-a5f2-d3265a643ee8 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:15.034529 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a099d452-dec8-47b9-a5f2-d3265a643ee8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:15.158593 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099d452-dec8-47b9-a5f2-d3265a643ee8 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:15.164165 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47118ca6-145c-40e4-be2d-3566e26ed7c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:15.165116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47118ca6-145c-40e4-be2d-3566e26ed7c6 service placement] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:15.165560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47118ca6-145c-40e4-be2d-3566e26ed7c6 service placement] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:15.165706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47118ca6-145c-40e4-be2d-3566e26ed7c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:15.168565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-47118ca6-145c-40e4-be2d-3566e26ed7c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:15.181486 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47118ca6-145c-40e4-be2d-3566e26ed7c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:15.183213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47118ca6-145c-40e4-be2d-3566e26ed7c6 service placement] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:15.194364 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1213/2422] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:16.011245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a019e7-3f0b-4f65-9c94-4b4c79587821 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:16.096287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a019e7-3f0b-4f65-9c94-4b4c79587821 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:16.101564 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:16.102774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:16.103282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:16.103762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] PATH_INFO: `/v3/users/d41d5c772dd74e38bef091682f399b20` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:16.105141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41d5c772dd74e38bef091682f399b20)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:16.107268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:16.117753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:16.520485 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41d5c772dd74e38bef091682f399b20'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:16.742656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41d5c772dd74e38bef091682f399b20'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:16.743058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41d5c772dd74e38bef091682f399b20'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:16.752915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41d5c772dd74e38bef091682f399b20'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:16.753661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41d5c772dd74e38bef091682f399b20'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:16.779309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1210/2423] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:16 2022] DELETE /identity/v3/users/d41d5c772dd74e38bef091682f399b20 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:16.782883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3fde5-a0f3-4fc7-8c5a-628c889ad3b7 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:16.782883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3fde5-a0f3-4fc7-8c5a-628c889ad3b7 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:16.782883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3fde5-a0f3-4fc7-8c5a-628c889ad3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:16.803889 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-23b3fde5-a0f3-4fc7-8c5a-628c889ad3b7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:16.805146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-23b3fde5-a0f3-4fc7-8c5a-628c889ad3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:16.931762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1214/2424] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:16.943297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:16.944969 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:17.093258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9742d9-00b8-4223-b59d-80ca9f28846c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:17.100899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c245e-c9fb-4db1-b377-d11bedfd3161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:17.101989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c245e-c9fb-4db1-b377-d11bedfd3161 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:17.102318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c245e-c9fb-4db1-b377-d11bedfd3161 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:17.107219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c245e-c9fb-4db1-b377-d11bedfd3161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:17.114424 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c245e-c9fb-4db1-b377-d11bedfd3161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:17.115985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c245e-c9fb-4db1-b377-d11bedfd3161 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:17.127628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1211/2425] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:17.841810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3fde5-a0f3-4fc7-8c5a-628c889ad3b7 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:17.843464 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-23b3fde5-a0f3-4fc7-8c5a-628c889ad3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:17.855828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3fde5-a0f3-4fc7-8c5a-628c889ad3b7 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:17.862309 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:17.863206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:17.863685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:17.864161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] PATH_INFO: `/v3/projects/28b50e8310394184a07533a5d49cc2c9` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:17.865668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b50e8310394184a07533a5d49cc2c9)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:17.867807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:18.054967 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b50e8310394184a07533a5d49cc2c9', 'request_id': 'req-5977845c-8e8f-4296-ae11-812210e2668f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.061591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b50e8310394184a07533a5d49cc2c9', 'request_id': 'req-5977845c-8e8f-4296-ae11-812210e2668f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.185369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b50e8310394184a07533a5d49cc2c9', 'request_id': 'req-5977845c-8e8f-4296-ae11-812210e2668f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.187270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:18.201555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] The token cache is being invalidate because project 28b50e8310394184a07533a5d49cc2c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:18.201901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.202563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.204102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1215/2426] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:17 2022] DELETE /identity/v3/projects/28b50e8310394184a07533a5d49cc2c9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:18.208162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c245e-c9fb-4db1-b377-d11bedfd3161 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:18.220888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c245e-c9fb-4db1-b377-d11bedfd3161 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:18.299563 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:18.300470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:18.300751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:18.301024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] PATH_INFO: `/v3/users/27e7540e7b844fecad728d20f5266228` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:18.302293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e7540e7b844fecad728d20f5266228)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:18.304242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:18.315964 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:18.645051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e7540e7b844fecad728d20f5266228'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.753670 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e7540e7b844fecad728d20f5266228'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.764025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e7540e7b844fecad728d20f5266228'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.764025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e7540e7b844fecad728d20f5266228'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.764025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7aebb285-b4e7-4aed-912c-f66dd940d729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e7540e7b844fecad728d20f5266228'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:18.781545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1212/2427] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:18 2022] DELETE /identity/v3/users/27e7540e7b844fecad728d20f5266228 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:19.457051 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:19.458638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:19.472241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977845c-8e8f-4296-ae11-812210e2668f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:19.542056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:19.543554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:19.543940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:19.544206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] PATH_INFO: `/v3/projects/ba905b9930ff4b9ea10f400b2b70111a` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:19.545526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba905b9930ff4b9ea10f400b2b70111a)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:19.547805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:19.659356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba905b9930ff4b9ea10f400b2b70111a', 'request_id': 'req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:19.675628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba905b9930ff4b9ea10f400b2b70111a', 'request_id': 'req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:19.780989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba905b9930ff4b9ea10f400b2b70111a', 'request_id': 'req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:19.793645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] The token cache is being invalidate because project ba905b9930ff4b9ea10f400b2b70111a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:19.793988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:19.794635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-81d7704f-feaf-4cc3-90a2-ffa07fc8ab63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:19.796237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1216/2428] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:19 2022] DELETE /identity/v3/projects/ba905b9930ff4b9ea10f400b2b70111a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:19.802650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08486-387a-4098-a1ae-a67fbe2498f9 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:19.803215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08486-387a-4098-a1ae-a67fbe2498f9 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:19.803317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08486-387a-4098-a1ae-a67fbe2498f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:19.821484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a4e08486-387a-4098-a1ae-a67fbe2498f9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:19.822755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a4e08486-387a-4098-a1ae-a67fbe2498f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:19.887174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1213/2429] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:19.895739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e791f-532a-49c2-8141-22e0973ab680 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:19.901172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e791f-532a-49c2-8141-22e0973ab680 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:19.901172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e791f-532a-49c2-8141-22e0973ab680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:19.915683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-719e791f-532a-49c2-8141-22e0973ab680 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:19.916909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-719e791f-532a-49c2-8141-22e0973ab680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:19.925857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1217/2430] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:19.930084 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08486-387a-4098-a1ae-a67fbe2498f9 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:19.940146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08486-387a-4098-a1ae-a67fbe2498f9 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:19.944858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722190cf-f392-4a8f-9a2a-bc30f281ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:19.945483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722190cf-f392-4a8f-9a2a-bc30f281ff11 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:19.945765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722190cf-f392-4a8f-9a2a-bc30f281ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:19.946083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722190cf-f392-4a8f-9a2a-bc30f281ff11 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:19.946876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722190cf-f392-4a8f-9a2a-bc30f281ff11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:19.947194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722190cf-f392-4a8f-9a2a-bc30f281ff11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:19.949435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722190cf-f392-4a8f-9a2a-bc30f281ff11 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:19.962045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1214/2431] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:03:19.967388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e791f-532a-49c2-8141-22e0973ab680 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:19.977339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e791f-532a-49c2-8141-22e0973ab680 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:19.982214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f7007-97ad-4d94-b0c8-fcb92f35dc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:19.982882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f7007-97ad-4d94-b0c8-fcb92f35dc9f admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:19.983171 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f7007-97ad-4d94-b0c8-fcb92f35dc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:19.983451 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f7007-97ad-4d94-b0c8-fcb92f35dc9f admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:19.984189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f7007-97ad-4d94-b0c8-fcb92f35dc9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:19.986148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f7007-97ad-4d94-b0c8-fcb92f35dc9f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:20.097576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1218/2432] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:19 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:20.101842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722190cf-f392-4a8f-9a2a-bc30f281ff11 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:20.112586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722190cf-f392-4a8f-9a2a-bc30f281ff11 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:20.175218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046138d7-293e-4b80-92c8-700a693eae59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:20.175829 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046138d7-293e-4b80-92c8-700a693eae59 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:20.176115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046138d7-293e-4b80-92c8-700a693eae59 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:20.176393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046138d7-293e-4b80-92c8-700a693eae59 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:20.177102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046138d7-293e-4b80-92c8-700a693eae59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:20.179149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046138d7-293e-4b80-92c8-700a693eae59 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:20.285136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1215/2433] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:20 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:20.289981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f7007-97ad-4d94-b0c8-fcb92f35dc9f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:20.306198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f7007-97ad-4d94-b0c8-fcb92f35dc9f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:20.311273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12382eaa-922a-427f-9fe0-7e4b3f0eb2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:20.312095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12382eaa-922a-427f-9fe0-7e4b3f0eb2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:20.314005 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12382eaa-922a-427f-9fe0-7e4b3f0eb2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:20.316417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12382eaa-922a-427f-9fe0-7e4b3f0eb2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:20.316417 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12382eaa-922a-427f-9fe0-7e4b3f0eb2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:20.316701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12382eaa-922a-427f-9fe0-7e4b3f0eb2a4 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:20.327774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1219/2434] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:20.331986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046138d7-293e-4b80-92c8-700a693eae59 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:20.338538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:20.343049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046138d7-293e-4b80-92c8-700a693eae59 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:20.347819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b953e-b8fb-4608-a8dd-17d77d8b956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:20.348513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b953e-b8fb-4608-a8dd-17d77d8b956a admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:20.348798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b953e-b8fb-4608-a8dd-17d77d8b956a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:20.349080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b953e-b8fb-4608-a8dd-17d77d8b956a admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:20.349865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b953e-b8fb-4608-a8dd-17d77d8b956a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:20.351905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b953e-b8fb-4608-a8dd-17d77d8b956a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:20.365363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1216/2435] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:20.370278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12382eaa-922a-427f-9fe0-7e4b3f0eb2a4 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:20.380738 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12382eaa-922a-427f-9fe0-7e4b3f0eb2a4 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:20.385448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198ffc-deb1-4788-99d8-95fd7e067835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:20.386388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198ffc-deb1-4788-99d8-95fd7e067835 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:20.386787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198ffc-deb1-4788-99d8-95fd7e067835 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:20.386948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198ffc-deb1-4788-99d8-95fd7e067835 admin admin] PATH_INFO: `/v3/projects/d6838a902386476599de69c19da8d2c7/users/bce09b344bf14ddd94983f1c52197a4b/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:20.406590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca198ffc-deb1-4788-99d8-95fd7e067835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6838a902386476599de69c19da8d2c7, user_id=bce09b344bf14ddd94983f1c52197a4b, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:20.409320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca198ffc-deb1-4788-99d8-95fd7e067835 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:20.523302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1220/2436] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:20 2022] PUT /identity/v3/projects/d6838a902386476599de69c19da8d2c7/users/bce09b344bf14ddd94983f1c52197a4b/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:20.536745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5bd05-e9f3-4b77-adaf-69fd7272f0c9 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:20.537042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5bd05-e9f3-4b77-adaf-69fd7272f0c9 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:20.537323 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5bd05-e9f3-4b77-adaf-69fd7272f0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:20.566705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f3c5bd05-e9f3-4b77-adaf-69fd7272f0c9 None None] MFA Rules not processed for user `bce09b344bf14ddd94983f1c52197a4b`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:20.567929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f3c5bd05-e9f3-4b77-adaf-69fd7272f0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:20.646569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1217/2437] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:20.659157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198ffc-deb1-4788-99d8-95fd7e067835 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:20.661700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ca198ffc-deb1-4788-99d8-95fd7e067835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:20.690017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:20.759633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198ffc-deb1-4788-99d8-95fd7e067835 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:20.765234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6402a-e546-4631-8fd0-226b5f8411d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:20.765606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6402a-e546-4631-8fd0-226b5f8411d4 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:20.765953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6402a-e546-4631-8fd0-226b5f8411d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:20.766237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6402a-e546-4631-8fd0-226b5f8411d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:20.776162 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6402a-e546-4631-8fd0-226b5f8411d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:20.778590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6402a-e546-4631-8fd0-226b5f8411d4 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:20.789779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1221/2438] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:20.822587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5bd05-e9f3-4b77-adaf-69fd7272f0c9 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:20.906254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5bd05-e9f3-4b77-adaf-69fd7272f0c9 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:20.911012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:20.911903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:20.912188 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:20.912464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] PATH_INFO: `/v3/users/bce09b344bf14ddd94983f1c52197a4b` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:20.913925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce09b344bf14ddd94983f1c52197a4b)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:20.915884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:21.191583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce09b344bf14ddd94983f1c52197a4b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:21.263576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce09b344bf14ddd94983f1c52197a4b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:21.264289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce09b344bf14ddd94983f1c52197a4b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:21.272794 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce09b344bf14ddd94983f1c52197a4b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:21.273654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce09b344bf14ddd94983f1c52197a4b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:21.293392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1218/2439] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:20 2022] DELETE /identity/v3/users/bce09b344bf14ddd94983f1c52197a4b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:21.298229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e44f01-418a-4a68-8b1f-10c0e16849b1 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:21.298525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e44f01-418a-4a68-8b1f-10c0e16849b1 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:21.298805 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e44f01-418a-4a68-8b1f-10c0e16849b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:21.318355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d4e44f01-418a-4a68-8b1f-10c0e16849b1 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:21.319583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d4e44f01-418a-4a68-8b1f-10c0e16849b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:21.389243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1222/2440] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:21.399219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:21.400711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:21.406155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:21.463711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdc509-cba5-44d7-b994-281515e88704 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:21.468268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116543cd-acd6-4a87-b3ee-b3e074ecbff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:21.468945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116543cd-acd6-4a87-b3ee-b3e074ecbff5 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:21.469263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116543cd-acd6-4a87-b3ee-b3e074ecbff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:21.469544 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116543cd-acd6-4a87-b3ee-b3e074ecbff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:21.479252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116543cd-acd6-4a87-b3ee-b3e074ecbff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:21.481237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116543cd-acd6-4a87-b3ee-b3e074ecbff5 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:21.490992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1219/2441] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:22.034964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e44f01-418a-4a68-8b1f-10c0e16849b1 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:22.036499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d4e44f01-418a-4a68-8b1f-10c0e16849b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:22.057158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e44f01-418a-4a68-8b1f-10c0e16849b1 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:22.062690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:22.064056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:22.064646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:22.064926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] PATH_INFO: `/v3/projects/d6838a902386476599de69c19da8d2c7` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:22.066963 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6838a902386476599de69c19da8d2c7)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:22.068995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:22.212439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6838a902386476599de69c19da8d2c7', 'request_id': 'req-7f2480fe-edbb-448a-9219-6fbab2067558'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:22.218799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6838a902386476599de69c19da8d2c7', 'request_id': 'req-7f2480fe-edbb-448a-9219-6fbab2067558'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:22.362553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6838a902386476599de69c19da8d2c7', 'request_id': 'req-7f2480fe-edbb-448a-9219-6fbab2067558'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:22.377967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116543cd-acd6-4a87-b3ee-b3e074ecbff5 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:22.379649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] The token cache is being invalidate because project d6838a902386476599de69c19da8d2c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:22.379933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:22.380637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:22.382327 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1223/2442] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:22 2022] DELETE /identity/v3/projects/d6838a902386476599de69c19da8d2c7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:22.395201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116543cd-acd6-4a87-b3ee-b3e074ecbff5 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:22.471222 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:22.472135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:22.472434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:22.472933 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:22.475735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:22.560504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:22.562013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:22.573957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1220/2443] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:03:23.920662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:23.922591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:23.935643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2480fe-edbb-448a-9219-6fbab2067558 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:23.940076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f83e7-8234-4dd3-b7e3-da3e3be3913a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:23.940630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f83e7-8234-4dd3-b7e3-da3e3be3913a service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:23.940926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f83e7-8234-4dd3-b7e3-da3e3be3913a service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:23.941234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f83e7-8234-4dd3-b7e3-da3e3be3913a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:23.943089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a75f83e7-8234-4dd3-b7e3-da3e3be3913a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:23.957595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f83e7-8234-4dd3-b7e3-da3e3be3913a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:23.959095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f83e7-8234-4dd3-b7e3-da3e3be3913a service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:23.971523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1224/2444] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:28.665982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:28.669302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:28.681886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f736ce-babd-4b45-b4ec-3a99bff5d42f service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:28.686149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:28.687047 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:28.687333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:28.687613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] PATH_INFO: `/v3/users/c4fd1155ff11417db8c647f0aadf74a1` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:28.688846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4fd1155ff11417db8c647f0aadf74a1)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:28.690856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:28.949512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4fd1155ff11417db8c647f0aadf74a1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:28.952067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:29.067106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4fd1155ff11417db8c647f0aadf74a1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:29.067776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4fd1155ff11417db8c647f0aadf74a1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:29.076461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4fd1155ff11417db8c647f0aadf74a1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:29.077905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-4b2a4833-0560-4a99-a034-f10f9c5157ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4fd1155ff11417db8c647f0aadf74a1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:29.103604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1221/2445] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:28 2022] DELETE /identity/v3/users/c4fd1155ff11417db8c647f0aadf74a1 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:29.475747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f83e7-8234-4dd3-b7e3-da3e3be3913a service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:29.494197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f83e7-8234-4dd3-b7e3-da3e3be3913a service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:29.507225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:29.587599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:29.588629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:29.589070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:29.589480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] PATH_INFO: `/v3/projects/975f7cf0a1f5445cb45ea441b3c056d5` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:29.591042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] RBAC: Authorizing `identity:delete_project(project_id=975f7cf0a1f5445cb45ea441b3c056d5)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:29.593041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:29.699157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '975f7cf0a1f5445cb45ea441b3c056d5', 'request_id': 'req-9a95d892-45d6-49a3-9e09-9b684c4a761d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:29.705986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '975f7cf0a1f5445cb45ea441b3c056d5', 'request_id': 'req-9a95d892-45d6-49a3-9e09-9b684c4a761d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:29.786899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '975f7cf0a1f5445cb45ea441b3c056d5', 'request_id': 'req-9a95d892-45d6-49a3-9e09-9b684c4a761d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:29.801649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] The token cache is being invalidate because project 975f7cf0a1f5445cb45ea441b3c056d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:29.805143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:29.805143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-9a95d892-45d6-49a3-9e09-9b684c4a761d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:29.805143 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1225/2446] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:29 2022] DELETE /identity/v3/projects/975f7cf0a1f5445cb45ea441b3c056d5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:29.810040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9f9a5-ff10-4e59-979f-74843a4ae02a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:29.811071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9f9a5-ff10-4e59-979f-74843a4ae02a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:29.811519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9f9a5-ff10-4e59-979f-74843a4ae02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:29.830672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-8fe9f9a5-ff10-4e59-979f-74843a4ae02a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:29.833679 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8fe9f9a5-ff10-4e59-979f-74843a4ae02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:29.904136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1222/2447] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:29.912580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39c86b-9e7e-4c8d-b17c-b1afded8d312 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:29.912920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39c86b-9e7e-4c8d-b17c-b1afded8d312 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:29.913208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39c86b-9e7e-4c8d-b17c-b1afded8d312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:29.934644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-2e39c86b-9e7e-4c8d-b17c-b1afded8d312 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:29.935911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2e39c86b-9e7e-4c8d-b17c-b1afded8d312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:29.944355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1226/2448] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:29.948698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9f9a5-ff10-4e59-979f-74843a4ae02a None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:29.958542 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9f9a5-ff10-4e59-979f-74843a4ae02a None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:29.963397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fb4ec-9e27-4857-addd-ed3ce4a09507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:29.966349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fb4ec-9e27-4857-addd-ed3ce4a09507 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:29.966349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fb4ec-9e27-4857-addd-ed3ce4a09507 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:29.966349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fb4ec-9e27-4857-addd-ed3ce4a09507 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:29.966349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fb4ec-9e27-4857-addd-ed3ce4a09507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:29.966349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fb4ec-9e27-4857-addd-ed3ce4a09507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:29.967736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fb4ec-9e27-4857-addd-ed3ce4a09507 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:29.980519 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1223/2449] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:03:29.984917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39c86b-9e7e-4c8d-b17c-b1afded8d312 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:29.994374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39c86b-9e7e-4c8d-b17c-b1afded8d312 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:29.999501 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce0e3-80d4-49c8-bf54-5480993a5592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:30.003966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce0e3-80d4-49c8-bf54-5480993a5592 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:30.003966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce0e3-80d4-49c8-bf54-5480993a5592 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:30.003966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce0e3-80d4-49c8-bf54-5480993a5592 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:30.003966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cce0e3-80d4-49c8-bf54-5480993a5592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:30.003966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cce0e3-80d4-49c8-bf54-5480993a5592 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:30.147841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1227/2450] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:29 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:30.153387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fb4ec-9e27-4857-addd-ed3ce4a09507 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:30.167740 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fb4ec-9e27-4857-addd-ed3ce4a09507 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:30.244471 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10300204-0965-4cf6-99f1-64b0b6059efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:30.245699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10300204-0965-4cf6-99f1-64b0b6059efc admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:30.246021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10300204-0965-4cf6-99f1-64b0b6059efc admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:30.246304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10300204-0965-4cf6-99f1-64b0b6059efc admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:30.247019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10300204-0965-4cf6-99f1-64b0b6059efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:30.249096 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10300204-0965-4cf6-99f1-64b0b6059efc admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:30.335059 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1224/2451] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:30 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:30.338534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce0e3-80d4-49c8-bf54-5480993a5592 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:30.349139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce0e3-80d4-49c8-bf54-5480993a5592 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:30.353957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b65e0b-043a-4cce-8ae8-4b1e7605905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:30.355154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b65e0b-043a-4cce-8ae8-4b1e7605905e admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:30.355439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b65e0b-043a-4cce-8ae8-4b1e7605905e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:30.355718 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b65e0b-043a-4cce-8ae8-4b1e7605905e admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:30.362415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b65e0b-043a-4cce-8ae8-4b1e7605905e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:30.362415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b65e0b-043a-4cce-8ae8-4b1e7605905e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:30.369030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1228/2452] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:30.373968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10300204-0965-4cf6-99f1-64b0b6059efc admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:30.385345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10300204-0965-4cf6-99f1-64b0b6059efc admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:30.390187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b8733-a1c8-4d75-9732-2d267aa2aba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:30.390856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b8733-a1c8-4d75-9732-2d267aa2aba0 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:30.391136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b8733-a1c8-4d75-9732-2d267aa2aba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:30.391410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b8733-a1c8-4d75-9732-2d267aa2aba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:30.392137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b8733-a1c8-4d75-9732-2d267aa2aba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:30.393769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462b8733-a1c8-4d75-9732-2d267aa2aba0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:30.411106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1225/2453] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:30.414493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b65e0b-043a-4cce-8ae8-4b1e7605905e admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:30.414907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b8733-a1c8-4d75-9732-2d267aa2aba0 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:30.417543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-14b65e0b-043a-4cce-8ae8-4b1e7605905e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:30.426778 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b8733-a1c8-4d75-9732-2d267aa2aba0 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:30.433773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dd5c2-f5f9-4831-8bbc-80fbb5cf7d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:30.435058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd5c2-f5f9-4831-8bbc-80fbb5cf7d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:30.435419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd5c2-f5f9-4831-8bbc-80fbb5cf7d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:30.435767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd5c2-f5f9-4831-8bbc-80fbb5cf7d74 admin admin] PATH_INFO: `/v3/projects/44efe47898574c92b83f60091de26bda/users/6de39fbf9eed4f8b84d11976bd7f2357/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:30.444253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b65e0b-043a-4cce-8ae8-4b1e7605905e admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:30.450767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:30.451662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:30.451938 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:30.452427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] PATH_INFO: `/v3/users/dcdbbbf1b9944b09898fa8731184c845` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:30.455842 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcdbbbf1b9944b09898fa8731184c845)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:30.456145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dd5c2-f5f9-4831-8bbc-80fbb5cf7d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44efe47898574c92b83f60091de26bda, user_id=6de39fbf9eed4f8b84d11976bd7f2357, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:30.457993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:30.459426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dd5c2-f5f9-4831-8bbc-80fbb5cf7d74 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:30.564741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1226/2454] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:30 2022] PUT /identity/v3/projects/44efe47898574c92b83f60091de26bda/users/6de39fbf9eed4f8b84d11976bd7f2357/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:30.575445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ee32a-b1ea-4c05-9d2d-1d3a6cb16bee None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:30.575739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ee32a-b1ea-4c05-9d2d-1d3a6cb16bee None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:30.576035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ee32a-b1ea-4c05-9d2d-1d3a6cb16bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:30.596832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:30.607566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6d4ee32a-b1ea-4c05-9d2d-1d3a6cb16bee None None] MFA Rules not processed for user `6de39fbf9eed4f8b84d11976bd7f2357`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:30.608844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6d4ee32a-b1ea-4c05-9d2d-1d3a6cb16bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:30.641263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcdbbbf1b9944b09898fa8731184c845'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:30.676569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1227/2455] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:30.682019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ee32a-b1ea-4c05-9d2d-1d3a6cb16bee None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:30.754672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ee32a-b1ea-4c05-9d2d-1d3a6cb16bee None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:30.759734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf4fc9-5e06-4a98-abbe-37ba13a67082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:30.760495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf4fc9-5e06-4a98-abbe-37ba13a67082 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:30.761642 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf4fc9-5e06-4a98-abbe-37ba13a67082 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:30.761928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf4fc9-5e06-4a98-abbe-37ba13a67082 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:30.762838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf4fc9-5e06-4a98-abbe-37ba13a67082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:30.765154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf4fc9-5e06-4a98-abbe-37ba13a67082 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:30.916695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcdbbbf1b9944b09898fa8731184c845'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:30.917936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcdbbbf1b9944b09898fa8731184c845'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:30.923331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:30.926640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcdbbbf1b9944b09898fa8731184c845'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:30.926991 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcdbbbf1b9944b09898fa8731184c845'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:30.946196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1229/2456] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:30 2022] DELETE /identity/v3/users/dcdbbbf1b9944b09898fa8731184c845 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:31.052283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1228/2457] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:30 2022] POST /identity/v3/projects => generated 385 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:31.056405 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.058385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:31.137476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79ae4b-d039-4679-a66b-eec53e2d5474 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.197758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17635a2b-7299-4c66-af51-510ceba29eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.206169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17635a2b-7299-4c66-af51-510ceba29eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.206169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17635a2b-7299-4c66-af51-510ceba29eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.206169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17635a2b-7299-4c66-af51-510ceba29eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.206169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17635a2b-7299-4c66-af51-510ceba29eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:31.206169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17635a2b-7299-4c66-af51-510ceba29eb1 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:31.262842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf4fc9-5e06-4a98-abbe-37ba13a67082 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.264695 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-40bf4fc9-5e06-4a98-abbe-37ba13a67082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:31.277464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf4fc9-5e06-4a98-abbe-37ba13a67082 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.282282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.283056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.283326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.283603 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] PATH_INFO: `/v3/projects/64a86c4d5f1043ca93987ed2c77a3629` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.284890 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a86c4d5f1043ca93987ed2c77a3629)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:31.286878 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:31.347739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1230/2458] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:31 2022] POST /identity/v3/users => generated 299 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:31.351549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17635a2b-7299-4c66-af51-510ceba29eb1 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.362449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17635a2b-7299-4c66-af51-510ceba29eb1 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.369570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f521b9-bec2-4956-bcb6-9902c7d1186d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.371018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f521b9-bec2-4956-bcb6-9902c7d1186d admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.371304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f521b9-bec2-4956-bcb6-9902c7d1186d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.371583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f521b9-bec2-4956-bcb6-9902c7d1186d admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.372320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f521b9-bec2-4956-bcb6-9902c7d1186d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:31.374081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f521b9-bec2-4956-bcb6-9902c7d1186d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:31.385463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1231/2459] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:31.391001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f521b9-bec2-4956-bcb6-9902c7d1186d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.400537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f521b9-bec2-4956-bcb6-9902c7d1186d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.405585 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c659e-aed5-4629-b681-0589e2cc3e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.406541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c659e-aed5-4629-b681-0589e2cc3e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.407176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c659e-aed5-4629-b681-0589e2cc3e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.407292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c659e-aed5-4629-b681-0589e2cc3e7d admin admin] PATH_INFO: `/v3/projects/f67515d3b7224109aed9a89bee74b35d/users/741aa235e2114bd3893677a6a89d66ea/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.424517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c659e-aed5-4629-b681-0589e2cc3e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67515d3b7224109aed9a89bee74b35d, user_id=741aa235e2114bd3893677a6a89d66ea, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:31.427532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c659e-aed5-4629-b681-0589e2cc3e7d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:31.533666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1232/2460] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:31 2022] PUT /identity/v3/projects/f67515d3b7224109aed9a89bee74b35d/users/741aa235e2114bd3893677a6a89d66ea/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:31.543259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c659e-aed5-4629-b681-0589e2cc3e7d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.545599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a86c4d5f1043ca93987ed2c77a3629', 'request_id': 'req-9ce66d89-56bf-48f3-83dd-f78ab70863dc'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:31.554238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a86c4d5f1043ca93987ed2c77a3629', 'request_id': 'req-9ce66d89-56bf-48f3-83dd-f78ab70863dc'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:31.603019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:31.618272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c659e-aed5-4629-b681-0589e2cc3e7d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.640830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a86c4d5f1043ca93987ed2c77a3629', 'request_id': 'req-9ce66d89-56bf-48f3-83dd-f78ab70863dc'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:31.656036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] The token cache is being invalidate because project 64a86c4d5f1043ca93987ed2c77a3629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:31.656391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:31.657293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:31.661009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1229/2461] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:31 2022] DELETE /identity/v3/projects/64a86c4d5f1043ca93987ed2c77a3629 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:31.661009 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.661787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:31.684461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:31.684860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb54d1-7f3e-4cef-bb46-1ed6ec6299ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.685557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb54d1-7f3e-4cef-bb46-1ed6ec6299ff admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.685873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb54d1-7f3e-4cef-bb46-1ed6ec6299ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.686161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb54d1-7f3e-4cef-bb46-1ed6ec6299ff admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.686915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb54d1-7f3e-4cef-bb46-1ed6ec6299ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:31.688640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb54d1-7f3e-4cef-bb46-1ed6ec6299ff admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:31.698918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1233/2462] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:31.700223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ca189-16ab-4e0c-a29e-64779944b796 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.700515 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ca189-16ab-4e0c-a29e-64779944b796 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.700806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ca189-16ab-4e0c-a29e-64779944b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.719273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cf7ca189-16ab-4e0c-a29e-64779944b796 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:31.720508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cf7ca189-16ab-4e0c-a29e-64779944b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:31.728861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1234/2463] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:31.730389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ca189-16ab-4e0c-a29e-64779944b796 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.731882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cf7ca189-16ab-4e0c-a29e-64779944b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:31.733657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce66d89-56bf-48f3-83dd-f78ab70863dc admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.737547 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e16c9b-5a17-46ae-a2d3-c838666c94bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.738239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e16c9b-5a17-46ae-a2d3-c838666c94bf service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.738528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e16c9b-5a17-46ae-a2d3-c838666c94bf service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.738810 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e16c9b-5a17-46ae-a2d3-c838666c94bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.740635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e9e16c9b-5a17-46ae-a2d3-c838666c94bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:31.744095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ca189-16ab-4e0c-a29e-64779944b796 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.749334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210776c5-f008-47da-8a81-e6519dc5632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.750256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210776c5-f008-47da-8a81-e6519dc5632b admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.750550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210776c5-f008-47da-8a81-e6519dc5632b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.750828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210776c5-f008-47da-8a81-e6519dc5632b admin admin] PATH_INFO: `/v3/projects/f67515d3b7224109aed9a89bee74b35d/users/741aa235e2114bd3893677a6a89d66ea/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.752957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210776c5-f008-47da-8a81-e6519dc5632b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f67515d3b7224109aed9a89bee74b35d, user_id=741aa235e2114bd3893677a6a89d66ea, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:31.753106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e16c9b-5a17-46ae-a2d3-c838666c94bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:31.754520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e16c9b-5a17-46ae-a2d3-c838666c94bf service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:31.756340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210776c5-f008-47da-8a81-e6519dc5632b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:31.765130 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1230/2464] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:31.766456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae613-f3ea-4e9f-be35-a7cc90e24391 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.766748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae613-f3ea-4e9f-be35-a7cc90e24391 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.767033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ae613-f3ea-4e9f-be35-a7cc90e24391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.769301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1235/2465] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:31 2022] PUT /identity/v3/projects/f67515d3b7224109aed9a89bee74b35d/users/741aa235e2114bd3893677a6a89d66ea/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:31.781582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcbe46-be04-44c6-b36f-a0dc1fb5a7a2 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.781941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcbe46-be04-44c6-b36f-a0dc1fb5a7a2 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.782229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcbe46-be04-44c6-b36f-a0dc1fb5a7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.787899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e18ae613-f3ea-4e9f-be35-a7cc90e24391 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:31.790699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e18ae613-f3ea-4e9f-be35-a7cc90e24391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:31.814564 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-99bcbe46-be04-44c6-b36f-a0dc1fb5a7a2 None None] MFA Rules not processed for user `741aa235e2114bd3893677a6a89d66ea`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:31.815953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-99bcbe46-be04-44c6-b36f-a0dc1fb5a7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:31.861915 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1231/2466] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:31.866525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ae613-f3ea-4e9f-be35-a7cc90e24391 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.876798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ae613-f3ea-4e9f-be35-a7cc90e24391 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.883404 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.884378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.884663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.884940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.886351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:31.886667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:31.888525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:31.898619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1236/2467] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:31.902666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1232/2468] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:03:31.907077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcbe46-be04-44c6-b36f-a0dc1fb5a7a2 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.911734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:31.915055 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:31.920931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcbe46-be04-44c6-b36f-a0dc1fb5a7a2 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.927808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abff0dd-cd09-4920-a9ca-205e7de85e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.933284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abff0dd-cd09-4920-a9ca-205e7de85e02 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.933284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abff0dd-cd09-4920-a9ca-205e7de85e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.933284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abff0dd-cd09-4920-a9ca-205e7de85e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:31.933284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abff0dd-cd09-4920-a9ca-205e7de85e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:31.933284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abff0dd-cd09-4920-a9ca-205e7de85e02 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:31.988529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436e18-631d-481f-b7a8-e5e703cd6214 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:31.992326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628222b-4334-4876-a63d-324d8b7d8ee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:31.993053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628222b-4334-4876-a63d-324d8b7d8ee5 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:31.993409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628222b-4334-4876-a63d-324d8b7d8ee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:31.993756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628222b-4334-4876-a63d-324d8b7d8ee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.003719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628222b-4334-4876-a63d-324d8b7d8ee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:32.005247 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628222b-4334-4876-a63d-324d8b7d8ee5 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:32.016494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1233/2469] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:32.057859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1237/2470] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:31 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:32.061627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628222b-4334-4876-a63d-324d8b7d8ee5 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:32.075164 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628222b-4334-4876-a63d-324d8b7d8ee5 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:32.095406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abff0dd-cd09-4920-a9ca-205e7de85e02 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:32.129100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=100 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:32.139859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:32.152836 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9f3a7-d254-4f17-8d58-a1e8c531d45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:32.153572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9f3a7-d254-4f17-8d58-a1e8c531d45c admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:32.153877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9f3a7-d254-4f17-8d58-a1e8c531d45c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:32.154159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9f3a7-d254-4f17-8d58-a1e8c531d45c admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.155010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9f3a7-d254-4f17-8d58-a1e8c531d45c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:32.157013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9f3a7-d254-4f17-8d58-a1e8c531d45c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:32.188272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abff0dd-cd09-4920-a9ca-205e7de85e02 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:32.193426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:32.194467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:32.194754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:32.195034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] PATH_INFO: `/v3/users/6de39fbf9eed4f8b84d11976bd7f2357` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.196282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de39fbf9eed4f8b84d11976bd7f2357)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:32.198349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:32.275968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1234/2471] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:32 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:32.277782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9f3a7-d254-4f17-8d58-a1e8c531d45c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:32.288689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9f3a7-d254-4f17-8d58-a1e8c531d45c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:32.294451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc4550-8fc9-45cf-8d3f-6c791cdc7ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:32.295124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc4550-8fc9-45cf-8d3f-6c791cdc7ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:32.295408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc4550-8fc9-45cf-8d3f-6c791cdc7ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:32.295754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc4550-8fc9-45cf-8d3f-6c791cdc7ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.296495 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc4550-8fc9-45cf-8d3f-6c791cdc7ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:32.298406 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc4550-8fc9-45cf-8d3f-6c791cdc7ec2 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:32.309678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1235/2472] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:32.314197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc4550-8fc9-45cf-8d3f-6c791cdc7ec2 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:32.326041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc4550-8fc9-45cf-8d3f-6c791cdc7ec2 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:32.331353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c93390-63bb-469b-b258-033893ce67a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:32.332035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c93390-63bb-469b-b258-033893ce67a5 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:32.332321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c93390-63bb-469b-b258-033893ce67a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:32.332605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c93390-63bb-469b-b258-033893ce67a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.333345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c93390-63bb-469b-b258-033893ce67a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:32.335004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c93390-63bb-469b-b258-033893ce67a5 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:32.348952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1236/2473] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:32.350028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c93390-63bb-469b-b258-033893ce67a5 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:32.360516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c93390-63bb-469b-b258-033893ce67a5 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:32.365274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2bba6-ec4a-441f-8905-f37b5ba6ff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:32.366210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2bba6-ec4a-441f-8905-f37b5ba6ff99 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:32.366498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2bba6-ec4a-441f-8905-f37b5ba6ff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:32.366784 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2bba6-ec4a-441f-8905-f37b5ba6ff99 admin admin] PATH_INFO: `/v3/projects/f8b4cd08a59a47a1ae0180ae4940896f/users/cbca492eda35499bae86a1346374b8de/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.383481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2bba6-ec4a-441f-8905-f37b5ba6ff99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b4cd08a59a47a1ae0180ae4940896f, user_id=cbca492eda35499bae86a1346374b8de, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:32.386809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2bba6-ec4a-441f-8905-f37b5ba6ff99 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:32.419783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de39fbf9eed4f8b84d11976bd7f2357'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:32.498373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1237/2474] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:32 2022] PUT /identity/v3/projects/f8b4cd08a59a47a1ae0180ae4940896f/users/cbca492eda35499bae86a1346374b8de/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:32.509566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cf4ab-905e-48ba-aa01-c8c21e179d23 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:32.509566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cf4ab-905e-48ba-aa01-c8c21e179d23 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:32.509566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cf4ab-905e-48ba-aa01-c8c21e179d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.514894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de39fbf9eed4f8b84d11976bd7f2357'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:32.514894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de39fbf9eed4f8b84d11976bd7f2357'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:32.524132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de39fbf9eed4f8b84d11976bd7f2357'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:32.524857 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-635ba298-b5a2-4761-b3a5-eb272126eddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de39fbf9eed4f8b84d11976bd7f2357'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:32.540008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-b61cf4ab-905e-48ba-aa01-c8c21e179d23 None None] MFA Rules not processed for user `cbca492eda35499bae86a1346374b8de`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:32.541623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b61cf4ab-905e-48ba-aa01-c8c21e179d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:32.548502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1238/2475] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:32 2022] DELETE /identity/v3/users/6de39fbf9eed4f8b84d11976bd7f2357 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:32.553585 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07665b-e77c-4132-8bf7-e0bbee701885 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:32.554000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07665b-e77c-4132-8bf7-e0bbee701885 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:32.554352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07665b-e77c-4132-8bf7-e0bbee701885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.574394 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-4b07665b-e77c-4132-8bf7-e0bbee701885 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:32.575815 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4b07665b-e77c-4132-8bf7-e0bbee701885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:32.631665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1238/2476] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:32.646075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1239/2477] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:32.647201 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cf4ab-905e-48ba-aa01-c8c21e179d23 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:32.649809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b61cf4ab-905e-48ba-aa01-c8c21e179d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:32.657765 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07665b-e77c-4132-8bf7-e0bbee701885 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:32.659248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4b07665b-e77c-4132-8bf7-e0bbee701885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:32.713676 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:32.726355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07665b-e77c-4132-8bf7-e0bbee701885 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:32.730215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:32.730361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccda465-6e67-4fa1-844f-9d64afbf98f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:32.730839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccda465-6e67-4fa1-844f-9d64afbf98f7 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:32.731117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccda465-6e67-4fa1-844f-9d64afbf98f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:32.731396 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccda465-6e67-4fa1-844f-9d64afbf98f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.740947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccda465-6e67-4fa1-844f-9d64afbf98f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:32.742457 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccda465-6e67-4fa1-844f-9d64afbf98f7 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:32.752306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1240/2478] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:32.768736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cf4ab-905e-48ba-aa01-c8c21e179d23 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:32.836811 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007d74c-1043-46b8-bb65-81fd43f90745 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:32.837494 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007d74c-1043-46b8-bb65-81fd43f90745 service nova] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:32.837788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007d74c-1043-46b8-bb65-81fd43f90745 service nova] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:32.838139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007d74c-1043-46b8-bb65-81fd43f90745 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:32.907074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007d74c-1043-46b8-bb65-81fd43f90745 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:32.908871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007d74c-1043-46b8-bb65-81fd43f90745 service nova] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:32.919712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1239/2479] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:33.237475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccda465-6e67-4fa1-844f-9d64afbf98f7 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:33.237475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cccda465-6e67-4fa1-844f-9d64afbf98f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:33.248420 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccda465-6e67-4fa1-844f-9d64afbf98f7 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:33.253679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:33.258877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:33.258877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:33.258877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] PATH_INFO: `/v3/users/cbca492eda35499bae86a1346374b8de` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:33.258877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbca492eda35499bae86a1346374b8de)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:33.258877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:33.428522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007d74c-1043-46b8-bb65-81fd43f90745 service nova] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:33.430333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c007d74c-1043-46b8-bb65-81fd43f90745 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:33.451293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007d74c-1043-46b8-bb65-81fd43f90745 service nova] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:33.455873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:33.456754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:33.457038 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:33.457682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] PATH_INFO: `/v3/projects/44efe47898574c92b83f60091de26bda` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:33.458995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] RBAC: Authorizing `identity:delete_project(project_id=44efe47898574c92b83f60091de26bda)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:33.460793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:33.685585 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbca492eda35499bae86a1346374b8de'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:33.688746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44efe47898574c92b83f60091de26bda', 'request_id': 'req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:33.696187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44efe47898574c92b83f60091de26bda', 'request_id': 'req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:33.838617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbca492eda35499bae86a1346374b8de'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:33.842494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbca492eda35499bae86a1346374b8de'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:33.848908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbca492eda35499bae86a1346374b8de'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:33.849587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-57e0ed1a-107a-4c92-b84a-5a3d713ad0e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbca492eda35499bae86a1346374b8de'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:33.868407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1241/2480] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:33 2022] DELETE /identity/v3/users/cbca492eda35499bae86a1346374b8de => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:33.872997 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98174f-2036-48ba-8e9e-60fe4410b708 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:33.873359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98174f-2036-48ba-8e9e-60fe4410b708 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:33.873708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98174f-2036-48ba-8e9e-60fe4410b708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:33.890797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ad98174f-2036-48ba-8e9e-60fe4410b708 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:33.893943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ad98174f-2036-48ba-8e9e-60fe4410b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:33.975494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1242/2481] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:33.986225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98174f-2036-48ba-8e9e-60fe4410b708 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:33.987704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ad98174f-2036-48ba-8e9e-60fe4410b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:34.027522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44efe47898574c92b83f60091de26bda', 'request_id': 'req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:34.053273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] The token cache is being invalidate because project 44efe47898574c92b83f60091de26bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:34.054256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:34.055172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:34.058994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1240/2482] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:33 2022] DELETE /identity/v3/projects/44efe47898574c92b83f60091de26bda => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:34.063309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:34.065772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:34.074690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:34.079437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98174f-2036-48ba-8e9e-60fe4410b708 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:34.082808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:34.083757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:34.084233 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:34.084706 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:34.087081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:34.102103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:34.103846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:34.122325 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1243/2483] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:34.165758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cf3bd-96ce-4981-9cc2-280a5353c93f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:34.172604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:34.173625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:34.173945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:34.174232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] PATH_INFO: `/v3/users/741aa235e2114bd3893677a6a89d66ea` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:34.175529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=741aa235e2114bd3893677a6a89d66ea)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:34.177513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:34.470129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '741aa235e2114bd3893677a6a89d66ea'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:34.535304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '741aa235e2114bd3893677a6a89d66ea'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:34.535952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '741aa235e2114bd3893677a6a89d66ea'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:34.545864 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '741aa235e2114bd3893677a6a89d66ea'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:34.548122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-6960800a-a34d-4411-9ba9-3fccb5ed8464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '741aa235e2114bd3893677a6a89d66ea'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:34.581061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1241/2484] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:34 2022] DELETE /identity/v3/users/741aa235e2114bd3893677a6a89d66ea => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:34.697009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:34.697427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:34.709026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:34.722574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2da1a-ac71-489a-9cf3-47884dab4bc0 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:34.798604 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:34.799513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:34.799803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:34.800078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] PATH_INFO: `/v3/projects/f8b4cd08a59a47a1ae0180ae4940896f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:34.801278 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b4cd08a59a47a1ae0180ae4940896f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:34.803415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:34.955620 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b4cd08a59a47a1ae0180ae4940896f', 'request_id': 'req-73223cfe-7f6a-4760-a377-76d834cb06e9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:34.967457 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b4cd08a59a47a1ae0180ae4940896f', 'request_id': 'req-73223cfe-7f6a-4760-a377-76d834cb06e9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:35.083934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b4cd08a59a47a1ae0180ae4940896f', 'request_id': 'req-73223cfe-7f6a-4760-a377-76d834cb06e9'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:35.105218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] The token cache is being invalidate because project f8b4cd08a59a47a1ae0180ae4940896f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:35.105523 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:35.106432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:35.109904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1244/2485] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:34 2022] DELETE /identity/v3/projects/f8b4cd08a59a47a1ae0180ae4940896f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:35.113232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0086af8-9c24-4186-9067-cd0da1cc7f54 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.113522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0086af8-9c24-4186-9067-cd0da1cc7f54 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.113804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0086af8-9c24-4186-9067-cd0da1cc7f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.134412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f0086af8-9c24-4186-9067-cd0da1cc7f54 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:35.134412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f0086af8-9c24-4186-9067-cd0da1cc7f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:35.196258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:35.198319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:35.217562 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1242/2486] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:35.223204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443616b-7afa-4571-9f1a-2add56ea2cdc None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.226781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443616b-7afa-4571-9f1a-2add56ea2cdc None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.226781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443616b-7afa-4571-9f1a-2add56ea2cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.250536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-6443616b-7afa-4571-9f1a-2add56ea2cdc None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:35.251914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6443616b-7afa-4571-9f1a-2add56ea2cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:35.261958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1243/2487] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:35.266772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443616b-7afa-4571-9f1a-2add56ea2cdc None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:35.284824 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443616b-7afa-4571-9f1a-2add56ea2cdc None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:35.297489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2d398-7f07-4ff2-b9b7-e915d8edc2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:35.298663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2d398-7f07-4ff2-b9b7-e915d8edc2ca admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.299977 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2d398-7f07-4ff2-b9b7-e915d8edc2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.300415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2d398-7f07-4ff2-b9b7-e915d8edc2ca admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.301787 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2d398-7f07-4ff2-b9b7-e915d8edc2ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:35.302293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2d398-7f07-4ff2-b9b7-e915d8edc2ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:35.305228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2d398-7f07-4ff2-b9b7-e915d8edc2ca admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:35.333910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1244/2488] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:03:35.334017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73223cfe-7f6a-4760-a377-76d834cb06e9 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:35.334118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2d398-7f07-4ff2-b9b7-e915d8edc2ca admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:35.340898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:35.341501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:35.342273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.342722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.343094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] PATH_INFO: `/v3/projects/f67515d3b7224109aed9a89bee74b35d` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.345389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f67515d3b7224109aed9a89bee74b35d)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:35.347632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:35.347957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2d398-7f07-4ff2-b9b7-e915d8edc2ca admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:35.366617 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cef11-12bb-4689-a768-96a16e6a0565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:35.367308 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cef11-12bb-4689-a768-96a16e6a0565 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.368100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cef11-12bb-4689-a768-96a16e6a0565 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.368301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cef11-12bb-4689-a768-96a16e6a0565 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.369018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cef11-12bb-4689-a768-96a16e6a0565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:35.372623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cef11-12bb-4689-a768-96a16e6a0565 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:35.539870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1245/2489] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:35 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:35.544229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cef11-12bb-4689-a768-96a16e6a0565 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:35.548680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:35.572776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f67515d3b7224109aed9a89bee74b35d', 'request_id': 'req-a67cd2be-39ad-48e4-a4f8-23b35822b868'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:35.581797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f67515d3b7224109aed9a89bee74b35d', 'request_id': 'req-a67cd2be-39ad-48e4-a4f8-23b35822b868'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:35.632559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cef11-12bb-4689-a768-96a16e6a0565 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:35.638386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbfe21-a1d1-4cc4-a923-b09d44719722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:35.639187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfe21-a1d1-4cc4-a923-b09d44719722 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.639381 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfe21-a1d1-4cc4-a923-b09d44719722 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.639658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfe21-a1d1-4cc4-a923-b09d44719722 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.640375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbfe21-a1d1-4cc4-a923-b09d44719722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:35.642529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbfe21-a1d1-4cc4-a923-b09d44719722 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:35.698699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f67515d3b7224109aed9a89bee74b35d', 'request_id': 'req-a67cd2be-39ad-48e4-a4f8-23b35822b868'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:35.714770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] The token cache is being invalidate because project f67515d3b7224109aed9a89bee74b35d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:35.715068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:35.716902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-a67cd2be-39ad-48e4-a4f8-23b35822b868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:35.720692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1245/2490] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:35 2022] DELETE /identity/v3/projects/f67515d3b7224109aed9a89bee74b35d => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:35.722158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f236cc4-a8d3-40f3-bfdf-8efc919c9813 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.722440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f236cc4-a8d3-40f3-bfdf-8efc919c9813 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.722714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f236cc4-a8d3-40f3-bfdf-8efc919c9813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.747714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-3f236cc4-a8d3-40f3-bfdf-8efc919c9813 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:35.749267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3f236cc4-a8d3-40f3-bfdf-8efc919c9813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:35.758262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1246/2491] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:35.768932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae01f6d-2867-4c38-8f19-d730e4d5c110 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.769311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae01f6d-2867-4c38-8f19-d730e4d5c110 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.769693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae01f6d-2867-4c38-8f19-d730e4d5c110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.792906 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-6ae01f6d-2867-4c38-8f19-d730e4d5c110 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:35.794470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6ae01f6d-2867-4c38-8f19-d730e4d5c110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:35.803727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1247/2492] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:35.809922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae01f6d-2867-4c38-8f19-d730e4d5c110 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:35.818887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1246/2493] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:35 2022] POST /identity/v3/users => generated 283 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:35.821957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae01f6d-2867-4c38-8f19-d730e4d5c110 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:35.822132 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbfe21-a1d1-4cc4-a923-b09d44719722 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:35.824321 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b0cbfe21-a1d1-4cc4-a923-b09d44719722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:35.829959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc300d-c484-4dd0-b746-61f29dad7ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:35.830945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc300d-c484-4dd0-b746-61f29dad7ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.831427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc300d-c484-4dd0-b746-61f29dad7ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.831872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc300d-c484-4dd0-b746-61f29dad7ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.832821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc300d-c484-4dd0-b746-61f29dad7ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:35.833307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc300d-c484-4dd0-b746-61f29dad7ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:35.835310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc300d-c484-4dd0-b746-61f29dad7ea7 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:35.842231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbfe21-a1d1-4cc4-a923-b09d44719722 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:35.853407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1248/2494] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:03:35.853677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e17ea-24c3-4ff1-888e-54b86b00fc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:35.867628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e17ea-24c3-4ff1-888e-54b86b00fc20 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.867628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e17ea-24c3-4ff1-888e-54b86b00fc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.867628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e17ea-24c3-4ff1-888e-54b86b00fc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.867628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e17ea-24c3-4ff1-888e-54b86b00fc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:35.867628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e17ea-24c3-4ff1-888e-54b86b00fc20 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:35.869030 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc300d-c484-4dd0-b746-61f29dad7ea7 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:35.882070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1247/2495] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:35.884536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e17ea-24c3-4ff1-888e-54b86b00fc20 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:35.886803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc300d-c484-4dd0-b746-61f29dad7ea7 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:35.896886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c467884-1753-44d2-b400-55371de2013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:35.897621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c467884-1753-44d2-b400-55371de2013b admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.903939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c467884-1753-44d2-b400-55371de2013b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.903939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c467884-1753-44d2-b400-55371de2013b admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.903939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c467884-1753-44d2-b400-55371de2013b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:35.903939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c467884-1753-44d2-b400-55371de2013b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:35.906615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e17ea-24c3-4ff1-888e-54b86b00fc20 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:35.913098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5cfee-09eb-4bee-a82b-4e37582c98ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:35.914008 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5cfee-09eb-4bee-a82b-4e37582c98ca admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.914366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5cfee-09eb-4bee-a82b-4e37582c98ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.914710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5cfee-09eb-4bee-a82b-4e37582c98ca admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.915738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5cfee-09eb-4bee-a82b-4e37582c98ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:35.918056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5cfee-09eb-4bee-a82b-4e37582c98ca admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:35.939210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1248/2496] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:35.939210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5cfee-09eb-4bee-a82b-4e37582c98ca admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:35.946499 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5cfee-09eb-4bee-a82b-4e37582c98ca admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:35.951459 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd17c95-3ccd-44ab-af85-43bc53609505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:35.952478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd17c95-3ccd-44ab-af85-43bc53609505 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:35.952832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd17c95-3ccd-44ab-af85-43bc53609505 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:35.953179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd17c95-3ccd-44ab-af85-43bc53609505 admin admin] PATH_INFO: `/v3/projects/d1aa191b2595405e9e27328c23c3faac/users/26110765d82c4d6494249db7c2e169f3/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:35.970746 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd17c95-3ccd-44ab-af85-43bc53609505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1aa191b2595405e9e27328c23c3faac, user_id=26110765d82c4d6494249db7c2e169f3, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:35.973539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd17c95-3ccd-44ab-af85-43bc53609505 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:36.103460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1249/2497] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:35 2022] POST /identity/v3/projects => generated 391 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:36.108602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c467884-1753-44d2-b400-55371de2013b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:36.139589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1249/2498] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:35 2022] PUT /identity/v3/projects/d1aa191b2595405e9e27328c23c3faac/users/26110765d82c4d6494249db7c2e169f3/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:36.151063 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a6160-e6dc-4c7b-bf1c-210c3c340635 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.151900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a6160-e6dc-4c7b-bf1c-210c3c340635 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.152158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a6160-e6dc-4c7b-bf1c-210c3c340635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.191608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e61a6160-e6dc-4c7b-bf1c-210c3c340635 None None] MFA Rules not processed for user `26110765d82c4d6494249db7c2e169f3`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:36.192747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e61a6160-e6dc-4c7b-bf1c-210c3c340635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:36.208537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c467884-1753-44d2-b400-55371de2013b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:36.292543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05d89d-3df8-4558-a2ad-6e689a56e1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:36.293913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05d89d-3df8-4558-a2ad-6e689a56e1af admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.294209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05d89d-3df8-4558-a2ad-6e689a56e1af admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.294880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05d89d-3df8-4558-a2ad-6e689a56e1af admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.295525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05d89d-3df8-4558-a2ad-6e689a56e1af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:36.298023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05d89d-3df8-4558-a2ad-6e689a56e1af admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:36.304612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1250/2499] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:36.309862 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a6160-e6dc-4c7b-bf1c-210c3c340635 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:36.321502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a6160-e6dc-4c7b-bf1c-210c3c340635 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:36.327812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d32e2-328d-4790-9459-d01cbb8cf0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:36.328453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d32e2-328d-4790-9459-d01cbb8cf0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.329000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d32e2-328d-4790-9459-d01cbb8cf0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.329275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d32e2-328d-4790-9459-d01cbb8cf0d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.330148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d32e2-328d-4790-9459-d01cbb8cf0d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:36.332179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d32e2-328d-4790-9459-d01cbb8cf0d4 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:36.386754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1250/2500] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:36 2022] POST /identity/v3/users => generated 296 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:36.390949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05d89d-3df8-4558-a2ad-6e689a56e1af admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:36.401215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05d89d-3df8-4558-a2ad-6e689a56e1af admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:36.406208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47048a-236a-48d9-bccb-2d92fabdf2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:36.406942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47048a-236a-48d9-bccb-2d92fabdf2ca admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.407227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47048a-236a-48d9-bccb-2d92fabdf2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.407610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47048a-236a-48d9-bccb-2d92fabdf2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.408331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47048a-236a-48d9-bccb-2d92fabdf2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:36.409955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47048a-236a-48d9-bccb-2d92fabdf2ca admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:36.420690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1251/2501] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:36.425585 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47048a-236a-48d9-bccb-2d92fabdf2ca admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:36.436375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47048a-236a-48d9-bccb-2d92fabdf2ca admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:36.441168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7d1d4-b17c-4303-b62e-7b885cc3ea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:36.445167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7d1d4-b17c-4303-b62e-7b885cc3ea4e admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.445167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7d1d4-b17c-4303-b62e-7b885cc3ea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.445167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7d1d4-b17c-4303-b62e-7b885cc3ea4e admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.445167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7d1d4-b17c-4303-b62e-7b885cc3ea4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:36.445167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7d1d4-b17c-4303-b62e-7b885cc3ea4e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:36.451432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1251/2502] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:36 2022] POST /identity/v3/projects => generated 363 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:36.455694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d32e2-328d-4790-9459-d01cbb8cf0d4 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:36.461498 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1252/2503] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:36.462913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7d1d4-b17c-4303-b62e-7b885cc3ea4e admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:36.472974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7d1d4-b17c-4303-b62e-7b885cc3ea4e admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:36.543780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d32e2-328d-4790-9459-d01cbb8cf0d4 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:36.551919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fe873-8932-42b3-94bf-03e971bdd45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:36.552530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fe873-8932-42b3-94bf-03e971bdd45a admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.552827 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fe873-8932-42b3-94bf-03e971bdd45a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.553345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fe873-8932-42b3-94bf-03e971bdd45a admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.553925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fe873-8932-42b3-94bf-03e971bdd45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:36.555993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fe873-8932-42b3-94bf-03e971bdd45a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:36.563734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c1314-36c3-44e1-bf2c-53a65398bfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:36.564908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c1314-36c3-44e1-bf2c-53a65398bfcc admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.565193 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c1314-36c3-44e1-bf2c-53a65398bfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.565468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c1314-36c3-44e1-bf2c-53a65398bfcc admin admin] PATH_INFO: `/v3/projects/0885fa6efe234605a49ef515fb51bc80/users/ba343822377b4993b0615d29ac304d86/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.575632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:36.583581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c1314-36c3-44e1-bf2c-53a65398bfcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0885fa6efe234605a49ef515fb51bc80, user_id=ba343822377b4993b0615d29ac304d86, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:36.586340 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c1314-36c3-44e1-bf2c-53a65398bfcc admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:36.689875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1252/2504] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:36 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:36.690738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fe873-8932-42b3-94bf-03e971bdd45a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:36.692430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a74fe873-8932-42b3-94bf-03e971bdd45a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:36.705091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fe873-8932-42b3-94bf-03e971bdd45a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:36.709742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:36.710645 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.710929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.711208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] PATH_INFO: `/v3/users/95c6d7130d6c4ae4b8460807e6820ef1` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.712408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c6d7130d6c4ae4b8460807e6820ef1)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:36.715041 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:36.737531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1253/2505] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:36 2022] PUT /identity/v3/projects/0885fa6efe234605a49ef515fb51bc80/users/ba343822377b4993b0615d29ac304d86/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:36.738543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c1314-36c3-44e1-bf2c-53a65398bfcc admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:36.808563 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c1314-36c3-44e1-bf2c-53a65398bfcc admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:36.814001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968b764-8c3d-4176-86ad-7a7cdfddc01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:36.814592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968b764-8c3d-4176-86ad-7a7cdfddc01f admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.815027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968b764-8c3d-4176-86ad-7a7cdfddc01f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.815524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968b764-8c3d-4176-86ad-7a7cdfddc01f admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.816266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968b764-8c3d-4176-86ad-7a7cdfddc01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:36.817907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968b764-8c3d-4176-86ad-7a7cdfddc01f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:36.829858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1254/2506] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:36.838725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd87c48-9c24-46ce-a2fb-8f4708b896a0 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.838725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd87c48-9c24-46ce-a2fb-8f4708b896a0 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.838725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd87c48-9c24-46ce-a2fb-8f4708b896a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.859616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-9fd87c48-9c24-46ce-a2fb-8f4708b896a0 None None] MFA Rules not processed for user `ba343822377b4993b0615d29ac304d86`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:36.860845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9fd87c48-9c24-46ce-a2fb-8f4708b896a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:36.920116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:36.931199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1255/2507] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:36.934759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd87c48-9c24-46ce-a2fb-8f4708b896a0 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:36.942547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd87c48-9c24-46ce-a2fb-8f4708b896a0 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:36.947492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b778970-5212-4e04-8bfc-a6ee8e850d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:36.948421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b778970-5212-4e04-8bfc-a6ee8e850d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:36.948695 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b778970-5212-4e04-8bfc-a6ee8e850d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:36.948966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b778970-5212-4e04-8bfc-a6ee8e850d9b admin admin] PATH_INFO: `/v3/projects/3bf17e540ab54819b06da1ae4dcd41d9/users/e49d56fe94e04428a3121060b80faab4/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:36.965075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b778970-5212-4e04-8bfc-a6ee8e850d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf17e540ab54819b06da1ae4dcd41d9, user_id=e49d56fe94e04428a3121060b80faab4, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:36.967756 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b778970-5212-4e04-8bfc-a6ee8e850d9b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.018195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c6d7130d6c4ae4b8460807e6820ef1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.101004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c6d7130d6c4ae4b8460807e6820ef1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.101702 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c6d7130d6c4ae4b8460807e6820ef1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.107901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1256/2508] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:36 2022] PUT /identity/v3/projects/3bf17e540ab54819b06da1ae4dcd41d9/users/e49d56fe94e04428a3121060b80faab4/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:37.112740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b778970-5212-4e04-8bfc-a6ee8e850d9b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.112907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c6d7130d6c4ae4b8460807e6820ef1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.113052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3b778970-5212-4e04-8bfc-a6ee8e850d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:37.113149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c6d7130d6c4ae4b8460807e6820ef1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.130278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1253/2509] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:36 2022] DELETE /identity/v3/users/95c6d7130d6c4ae4b8460807e6820ef1 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:37.131140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.132464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:37.188107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b778970-5212-4e04-8bfc-a6ee8e850d9b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.219857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629bd5e-823e-4ccd-ac17-6e733ebc74e2 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.224928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d222723-a5f4-48a4-b456-f2cec9f480d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.227326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d222723-a5f4-48a4-b456-f2cec9f480d4 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.227326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d222723-a5f4-48a4-b456-f2cec9f480d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.227326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d222723-a5f4-48a4-b456-f2cec9f480d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.227326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d222723-a5f4-48a4-b456-f2cec9f480d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.228615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d222723-a5f4-48a4-b456-f2cec9f480d4 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.240016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1254/2510] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:37.244045 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d222723-a5f4-48a4-b456-f2cec9f480d4 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.253506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22ad45-08b9-4f02-a9d2-d12315e6f73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.254309 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22ad45-08b9-4f02-a9d2-d12315e6f73c admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.254668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22ad45-08b9-4f02-a9d2-d12315e6f73c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.255021 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22ad45-08b9-4f02-a9d2-d12315e6f73c admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.255148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d222723-a5f4-48a4-b456-f2cec9f480d4 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.255886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22ad45-08b9-4f02-a9d2-d12315e6f73c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.257968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22ad45-08b9-4f02-a9d2-d12315e6f73c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.262019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1343f18-5f78-456a-82a7-857fc9c17282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.263568 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1343f18-5f78-456a-82a7-857fc9c17282 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.263853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1343f18-5f78-456a-82a7-857fc9c17282 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.264127 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1343f18-5f78-456a-82a7-857fc9c17282 admin admin] PATH_INFO: `/v3/projects/3bf17e540ab54819b06da1ae4dcd41d9/users/e49d56fe94e04428a3121060b80faab4/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.266525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1343f18-5f78-456a-82a7-857fc9c17282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf17e540ab54819b06da1ae4dcd41d9, user_id=e49d56fe94e04428a3121060b80faab4, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.270085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1343f18-5f78-456a-82a7-857fc9c17282 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.283037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1255/2511] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:37 2022] PUT /identity/v3/projects/3bf17e540ab54819b06da1ae4dcd41d9/users/e49d56fe94e04428a3121060b80faab4/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:37.295067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6647fa-4200-4ae1-b9d6-6da4e70e967e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.295497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6647fa-4200-4ae1-b9d6-6da4e70e967e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.296529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6647fa-4200-4ae1-b9d6-6da4e70e967e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.337403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-fb6647fa-4200-4ae1-b9d6-6da4e70e967e None None] MFA Rules not processed for user `e49d56fe94e04428a3121060b80faab4`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:37.338709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fb6647fa-4200-4ae1-b9d6-6da4e70e967e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:37.346453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1257/2512] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:37 2022] POST /identity/v3/projects => generated 391 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:37.347560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22ad45-08b9-4f02-a9d2-d12315e6f73c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.371450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:37.413360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22ad45-08b9-4f02-a9d2-d12315e6f73c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.417551 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1256/2513] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:37 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:37.418381 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153c387-902a-4c41-84b0-eb8ac846c99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.419184 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153c387-902a-4c41-84b0-eb8ac846c99d admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.419530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153c387-902a-4c41-84b0-eb8ac846c99d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.419902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153c387-902a-4c41-84b0-eb8ac846c99d admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.420659 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153c387-902a-4c41-84b0-eb8ac846c99d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.423179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153c387-902a-4c41-84b0-eb8ac846c99d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.429410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6647fa-4200-4ae1-b9d6-6da4e70e967e None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.431309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fb6647fa-4200-4ae1-b9d6-6da4e70e967e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:37.494338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6647fa-4200-4ae1-b9d6-6da4e70e967e None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.498575 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a756a2-f13c-4c39-9b96-09e53cd9524a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.499241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a756a2-f13c-4c39-9b96-09e53cd9524a service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.499529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a756a2-f13c-4c39-9b96-09e53cd9524a service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.499844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a756a2-f13c-4c39-9b96-09e53cd9524a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.509413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a756a2-f13c-4c39-9b96-09e53cd9524a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.512369 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a756a2-f13c-4c39-9b96-09e53cd9524a service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.521711 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1257/2514] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:37 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:03:37.522701 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a756a2-f13c-4c39-9b96-09e53cd9524a service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.524080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-13a756a2-f13c-4c39-9b96-09e53cd9524a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:37.538875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a756a2-f13c-4c39-9b96-09e53cd9524a service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.545769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1258/2515] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:37 2022] POST /identity/v3/users => generated 302 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:37.547085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.547085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.547085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.547085 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] PATH_INFO: `/v3/projects/6c247c5f4fb74c78baaf3f4a63a04e42` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.547818 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c247c5f4fb74c78baaf3f4a63a04e42)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.547917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153c387-902a-4c41-84b0-eb8ac846c99d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.549021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.557511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153c387-902a-4c41-84b0-eb8ac846c99d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.563097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708be161-7839-4588-b299-17ecb4f4b76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.563772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708be161-7839-4588-b299-17ecb4f4b76c admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.564111 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708be161-7839-4588-b299-17ecb4f4b76c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.564480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708be161-7839-4588-b299-17ecb4f4b76c admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.565602 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708be161-7839-4588-b299-17ecb4f4b76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.567235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708be161-7839-4588-b299-17ecb4f4b76c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.578207 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1259/2516] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:37.582357 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708be161-7839-4588-b299-17ecb4f4b76c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.592920 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708be161-7839-4588-b299-17ecb4f4b76c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.598004 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da37514-95a5-4ce1-86f2-a50b0d474987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.598928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da37514-95a5-4ce1-86f2-a50b0d474987 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.599237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da37514-95a5-4ce1-86f2-a50b0d474987 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.599525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da37514-95a5-4ce1-86f2-a50b0d474987 admin admin] PATH_INFO: `/v3/projects/a871a6b0b25c4ce3a25e9620dbcf6366/users/ed5fc043b4bd4a2bbd5e040bb0e80795/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.615944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da37514-95a5-4ce1-86f2-a50b0d474987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a871a6b0b25c4ce3a25e9620dbcf6366, user_id=ed5fc043b4bd4a2bbd5e040bb0e80795, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.618660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da37514-95a5-4ce1-86f2-a50b0d474987 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.689732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:37.751376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1260/2517] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:37 2022] PUT /identity/v3/projects/a871a6b0b25c4ce3a25e9620dbcf6366/users/ed5fc043b4bd4a2bbd5e040bb0e80795/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:37.755524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da37514-95a5-4ce1-86f2-a50b0d474987 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.768306 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c247c5f4fb74c78baaf3f4a63a04e42', 'request_id': 'req-e3730cc6-bef2-424b-b4ba-85a60dff78b8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.775735 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c247c5f4fb74c78baaf3f4a63a04e42', 'request_id': 'req-e3730cc6-bef2-424b-b4ba-85a60dff78b8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.838293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da37514-95a5-4ce1-86f2-a50b0d474987 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.843530 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c247c5f4fb74c78baaf3f4a63a04e42', 'request_id': 'req-e3730cc6-bef2-424b-b4ba-85a60dff78b8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.856237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] The token cache is being invalidate because project 6c247c5f4fb74c78baaf3f4a63a04e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:37.856515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.857239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:37.860433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1258/2518] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:37 2022] DELETE /identity/v3/projects/6c247c5f4fb74c78baaf3f4a63a04e42 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:37.863151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.864600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:37.884808 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:37.907639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff0ed3b-63d0-4e12-be37-14f1a7281d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.908198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0ed3b-63d0-4e12-be37-14f1a7281d01 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.908198 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0ed3b-63d0-4e12-be37-14f1a7281d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.908483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff0ed3b-63d0-4e12-be37-14f1a7281d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.909084 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0ed3b-63d0-4e12-be37-14f1a7281d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.910803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff0ed3b-63d0-4e12-be37-14f1a7281d01 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.925707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1261/2519] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:37.925707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c92ef-b134-4d26-a197-1464cc7f7588 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.925707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c92ef-b134-4d26-a197-1464cc7f7588 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.925707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c92ef-b134-4d26-a197-1464cc7f7588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.941003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3730cc6-bef2-424b-b4ba-85a60dff78b8 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.944229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-fa6c92ef-b134-4d26-a197-1464cc7f7588 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:37.944712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331ed97-749a-4b2a-b7f3-2a8208d6204b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.945370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331ed97-749a-4b2a-b7f3-2a8208d6204b service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.945474 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fa6c92ef-b134-4d26-a197-1464cc7f7588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:37.945785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331ed97-749a-4b2a-b7f3-2a8208d6204b service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.946098 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331ed97-749a-4b2a-b7f3-2a8208d6204b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.947997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6331ed97-749a-4b2a-b7f3-2a8208d6204b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:37.954161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:37.959328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1262/2520] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:37.960211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c92ef-b134-4d26-a197-1464cc7f7588 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:37.963779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fa6c92ef-b134-4d26-a197-1464cc7f7588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:37.973648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c92ef-b134-4d26-a197-1464cc7f7588 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:37.978806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426df4e8-f403-4534-a063-e6455866583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:37.980334 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426df4e8-f403-4534-a063-e6455866583a admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.980629 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426df4e8-f403-4534-a063-e6455866583a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.980907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426df4e8-f403-4534-a063-e6455866583a admin admin] PATH_INFO: `/v3/projects/a871a6b0b25c4ce3a25e9620dbcf6366/users/ed5fc043b4bd4a2bbd5e040bb0e80795/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:37.983255 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426df4e8-f403-4534-a063-e6455866583a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a871a6b0b25c4ce3a25e9620dbcf6366, user_id=ed5fc043b4bd4a2bbd5e040bb0e80795, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:37.986023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426df4e8-f403-4534-a063-e6455866583a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:37.997569 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1263/2521] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:37 2022] PUT /identity/v3/projects/a871a6b0b25c4ce3a25e9620dbcf6366/users/ed5fc043b4bd4a2bbd5e040bb0e80795/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:37.998898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078b697-dd16-4231-b200-ba67bc85fe90 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:37.999192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078b697-dd16-4231-b200-ba67bc85fe90 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:37.999470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078b697-dd16-4231-b200-ba67bc85fe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.019116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0078b697-dd16-4231-b200-ba67bc85fe90 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:38.020361 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0078b697-dd16-4231-b200-ba67bc85fe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:38.025142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331ed97-749a-4b2a-b7f3-2a8208d6204b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.026708 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331ed97-749a-4b2a-b7f3-2a8208d6204b service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.040911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1259/2522] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:38.042994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855f441-da67-4ed0-be39-6251c5d7e99c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.047226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855f441-da67-4ed0-be39-6251c5d7e99c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.047226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855f441-da67-4ed0-be39-6251c5d7e99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.078414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-c855f441-da67-4ed0-be39-6251c5d7e99c None None] MFA Rules not processed for user `ed5fc043b4bd4a2bbd5e040bb0e80795`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:38.079816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c855f441-da67-4ed0-be39-6251c5d7e99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:38.108611 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1264/2523] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:38.113042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078b697-dd16-4231-b200-ba67bc85fe90 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:38.123705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078b697-dd16-4231-b200-ba67bc85fe90 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:38.129951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb1ae-87ff-4871-a829-5604d4a885b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:38.132833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb1ae-87ff-4871-a829-5604d4a885b4 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.132833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb1ae-87ff-4871-a829-5604d4a885b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.132833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bb1ae-87ff-4871-a829-5604d4a885b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.132833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bb1ae-87ff-4871-a829-5604d4a885b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:38.132833 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bb1ae-87ff-4871-a829-5604d4a885b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.134103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bb1ae-87ff-4871-a829-5604d4a885b4 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.148597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1265/2524] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:03:38.153470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb1ae-87ff-4871-a829-5604d4a885b4 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:38.163739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bb1ae-87ff-4871-a829-5604d4a885b4 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:38.169456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dee9b4-4567-41df-b152-d065eaa94d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:38.170073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1260/2525] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:38.170251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dee9b4-4567-41df-b152-d065eaa94d14 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.170527 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dee9b4-4567-41df-b152-d065eaa94d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.170874 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dee9b4-4567-41df-b152-d065eaa94d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.171572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dee9b4-4567-41df-b152-d065eaa94d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.173862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dee9b4-4567-41df-b152-d065eaa94d14 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.181113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855f441-da67-4ed0-be39-6251c5d7e99c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:38.182511 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c855f441-da67-4ed0-be39-6251c5d7e99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:38.248651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855f441-da67-4ed0-be39-6251c5d7e99c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:38.252432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b8226-353d-4f90-9d28-4f1c45c21888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:38.253175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8226-353d-4f90-9d28-4f1c45c21888 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.253462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8226-353d-4f90-9d28-4f1c45c21888 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.253739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8226-353d-4f90-9d28-4f1c45c21888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.263754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b8226-353d-4f90-9d28-4f1c45c21888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.265197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b8226-353d-4f90-9d28-4f1c45c21888 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.276280 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1261/2526] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:03:38.297139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:38.310534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1266/2527] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:38 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:38.314705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b8226-353d-4f90-9d28-4f1c45c21888 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:38.325472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b8226-353d-4f90-9d28-4f1c45c21888 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:38.338202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dee9b4-4567-41df-b152-d065eaa94d14 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:38.393021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05d1c6-5aff-454f-ba82-3cb299ff8b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:38.393747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05d1c6-5aff-454f-ba82-3cb299ff8b9d admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.393946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dee9b4-4567-41df-b152-d065eaa94d14 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:38.400906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05d1c6-5aff-454f-ba82-3cb299ff8b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.400906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05d1c6-5aff-454f-ba82-3cb299ff8b9d admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.400906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05d1c6-5aff-454f-ba82-3cb299ff8b9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.400906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05d1c6-5aff-454f-ba82-3cb299ff8b9d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.401308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e2d6a-24b0-4aed-a543-7f2ecdfb919d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:38.401308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e2d6a-24b0-4aed-a543-7f2ecdfb919d service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.401308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e2d6a-24b0-4aed-a543-7f2ecdfb919d service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.401308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e2d6a-24b0-4aed-a543-7f2ecdfb919d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.402231 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-db6e2d6a-24b0-4aed-a543-7f2ecdfb919d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:38.463475 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e2d6a-24b0-4aed-a543-7f2ecdfb919d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.465722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e2d6a-24b0-4aed-a543-7f2ecdfb919d service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.474320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:38.475565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1267/2528] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:03:38.485215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1262/2529] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:38 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:38.486770 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e2d6a-24b0-4aed-a543-7f2ecdfb919d service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:38.505210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e2d6a-24b0-4aed-a543-7f2ecdfb919d service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:38.513256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dc887-31dd-4db1-bc2a-5c1929af966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:38.513996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dc887-31dd-4db1-bc2a-5c1929af966f admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.514273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dc887-31dd-4db1-bc2a-5c1929af966f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.514609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dc887-31dd-4db1-bc2a-5c1929af966f admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.515401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dc887-31dd-4db1-bc2a-5c1929af966f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.517077 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008dc887-31dd-4db1-bc2a-5c1929af966f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.529179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1268/2530] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:38.533020 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05d1c6-5aff-454f-ba82-3cb299ff8b9d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:38.549719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05d1c6-5aff-454f-ba82-3cb299ff8b9d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:38.555033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473615b5-7703-4534-a815-ae380f933cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:38.555844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473615b5-7703-4534-a815-ae380f933cdc admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.556858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473615b5-7703-4534-a815-ae380f933cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.556858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473615b5-7703-4534-a815-ae380f933cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.557218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473615b5-7703-4534-a815-ae380f933cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.559340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473615b5-7703-4534-a815-ae380f933cdc admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.572859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1263/2531] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:38.576522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dc887-31dd-4db1-bc2a-5c1929af966f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:38.590001 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008dc887-31dd-4db1-bc2a-5c1929af966f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:38.596859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d52d4-8cdd-4d9b-9448-dc733c1bb5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:38.597776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d52d4-8cdd-4d9b-9448-dc733c1bb5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.598096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d52d4-8cdd-4d9b-9448-dc733c1bb5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.598376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d52d4-8cdd-4d9b-9448-dc733c1bb5fa admin admin] PATH_INFO: `/v3/projects/b3099836957a40e793fe9a561204e9b3/users/73c370b8968f40618fbac9b3e010b39d/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.614046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d52d4-8cdd-4d9b-9448-dc733c1bb5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3099836957a40e793fe9a561204e9b3, user_id=73c370b8968f40618fbac9b3e010b39d, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.616688 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d52d4-8cdd-4d9b-9448-dc733c1bb5fa admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.731924 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1269/2532] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:38 2022] PUT /identity/v3/projects/b3099836957a40e793fe9a561204e9b3/users/73c370b8968f40618fbac9b3e010b39d/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:38.746140 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aef83-ce55-4605-982e-b11154dad67c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.746428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aef83-ce55-4605-982e-b11154dad67c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.746704 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5aef83-ce55-4605-982e-b11154dad67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.776064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-3f5aef83-ce55-4605-982e-b11154dad67c None None] MFA Rules not processed for user `73c370b8968f40618fbac9b3e010b39d`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:38.778181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3f5aef83-ce55-4605-982e-b11154dad67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:38.848871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1264/2533] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:38.860714 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d52d4-8cdd-4d9b-9448-dc733c1bb5fa admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:38.862412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0a4d52d4-8cdd-4d9b-9448-dc733c1bb5fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:38.902579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:38.941042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d52d4-8cdd-4d9b-9448-dc733c1bb5fa admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:38.946590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b100af-27c2-49a2-a036-af2720c0e52a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:38.947246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b100af-27c2-49a2-a036-af2720c0e52a service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:38.947532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b100af-27c2-49a2-a036-af2720c0e52a service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:38.947810 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b100af-27c2-49a2-a036-af2720c0e52a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:38.956656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b100af-27c2-49a2-a036-af2720c0e52a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:38.958279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b100af-27c2-49a2-a036-af2720c0e52a service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:38.967406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1270/2534] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:42.121553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5aef83-ce55-4605-982e-b11154dad67c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:42.124377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-3f5aef83-ce55-4605-982e-b11154dad67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:42.180290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:42.203761 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5aef83-ce55-4605-982e-b11154dad67c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:42.208870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:42.209749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:42.210056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:42.210375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] PATH_INFO: `/v3/users/26110765d82c4d6494249db7c2e169f3` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:42.211516 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] RBAC: Authorizing `identity:delete_user(user_id=26110765d82c4d6494249db7c2e169f3)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:42.213470 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:42.531057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26110765d82c4d6494249db7c2e169f3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:42.630540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26110765d82c4d6494249db7c2e169f3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:42.631195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26110765d82c4d6494249db7c2e169f3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:42.639284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26110765d82c4d6494249db7c2e169f3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:42.640031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26110765d82c4d6494249db7c2e169f3'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:42.658480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1265/2535] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:42 2022] DELETE /identity/v3/users/26110765d82c4d6494249db7c2e169f3 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:42.663256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8c1a4-8b01-43ba-af02-0c1ce2498b11 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:42.663749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8c1a4-8b01-43ba-af02-0c1ce2498b11 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:42.663887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8c1a4-8b01-43ba-af02-0c1ce2498b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:42.681904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-36f8c1a4-8b01-43ba-af02-0c1ce2498b11 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:42.683699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-36f8c1a4-8b01-43ba-af02-0c1ce2498b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:42.742650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1271/2536] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:42.761378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:42.764220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:42.824581 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614b966-3220-4719-a79c-84507ddce0da admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:42.828205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81603d-073d-4005-9750-bc70597accd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:42.828860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81603d-073d-4005-9750-bc70597accd9 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:42.829145 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81603d-073d-4005-9750-bc70597accd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:42.829427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81603d-073d-4005-9750-bc70597accd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:42.838869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81603d-073d-4005-9750-bc70597accd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:42.845638 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81603d-073d-4005-9750-bc70597accd9 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:42.850854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1266/2537] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:43.272291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8c1a4-8b01-43ba-af02-0c1ce2498b11 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:43.273844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-36f8c1a4-8b01-43ba-af02-0c1ce2498b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:43.290846 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8c1a4-8b01-43ba-af02-0c1ce2498b11 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:43.295763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:43.298047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:43.298047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:43.298047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] PATH_INFO: `/v3/projects/d1aa191b2595405e9e27328c23c3faac` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:43.298792 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1aa191b2595405e9e27328c23c3faac)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:43.300804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:43.444981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1aa191b2595405e9e27328c23c3faac', 'request_id': 'req-ec70971e-5e44-4d2f-8909-645a0e472dd3'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.451632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1aa191b2595405e9e27328c23c3faac', 'request_id': 'req-ec70971e-5e44-4d2f-8909-645a0e472dd3'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.534653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1aa191b2595405e9e27328c23c3faac', 'request_id': 'req-ec70971e-5e44-4d2f-8909-645a0e472dd3'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.547722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] The token cache is being invalidate because project d1aa191b2595405e9e27328c23c3faac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:43.548027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.548668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.550253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1272/2538] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:43 2022] DELETE /identity/v3/projects/d1aa191b2595405e9e27328c23c3faac => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:43.555166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81603d-073d-4005-9750-bc70597accd9 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:43.566017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81603d-073d-4005-9750-bc70597accd9 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:43.623612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:43.624484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:43.624761 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:43.625030 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] PATH_INFO: `/v3/users/e49d56fe94e04428a3121060b80faab4` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:43.626226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49d56fe94e04428a3121060b80faab4)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:43.628150 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:43.630492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:43.830097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49d56fe94e04428a3121060b80faab4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.904665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49d56fe94e04428a3121060b80faab4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.905311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49d56fe94e04428a3121060b80faab4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.913000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49d56fe94e04428a3121060b80faab4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.914615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-909e3b5a-6d7f-4d0c-a3d9-16783f2ea90a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49d56fe94e04428a3121060b80faab4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:43.931419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1267/2539] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:43 2022] DELETE /identity/v3/users/e49d56fe94e04428a3121060b80faab4 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:44.371208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:44.372663 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:44.386836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70971e-5e44-4d2f-8909-645a0e472dd3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:44.429161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:44.453532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:44.454530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:44.454816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:44.455087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] PATH_INFO: `/v3/projects/3bf17e540ab54819b06da1ae4dcd41d9` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:44.456263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf17e540ab54819b06da1ae4dcd41d9)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:44.458190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:44.565203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf17e540ab54819b06da1ae4dcd41d9', 'request_id': 'req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:44.570901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf17e540ab54819b06da1ae4dcd41d9', 'request_id': 'req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:44.643834 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf17e540ab54819b06da1ae4dcd41d9', 'request_id': 'req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:44.655965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] The token cache is being invalidate because project 3bf17e540ab54819b06da1ae4dcd41d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:44.656250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:44.656894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8978ca2c-9f80-4eaa-bc5f-ce628a782c42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:44.658490 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1273/2540] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:44 2022] DELETE /identity/v3/projects/3bf17e540ab54819b06da1ae4dcd41d9 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:44.663688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba942ded-0692-4871-84a0-d9fe285a2752 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:44.664200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba942ded-0692-4871-84a0-d9fe285a2752 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:44.664410 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba942ded-0692-4871-84a0-d9fe285a2752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:44.684159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ba942ded-0692-4871-84a0-d9fe285a2752 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:44.685387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ba942ded-0692-4871-84a0-d9fe285a2752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:44.747344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1268/2541] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:44.755470 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104df2a1-82cc-4ddb-8f4f-ea24d51a9194 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:44.755764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104df2a1-82cc-4ddb-8f4f-ea24d51a9194 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:44.756067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104df2a1-82cc-4ddb-8f4f-ea24d51a9194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:44.775446 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-104df2a1-82cc-4ddb-8f4f-ea24d51a9194 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:44.776720 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-104df2a1-82cc-4ddb-8f4f-ea24d51a9194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:44.785007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1274/2542] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:44.790241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba942ded-0692-4871-84a0-d9fe285a2752 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:44.800867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba942ded-0692-4871-84a0-d9fe285a2752 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:44.805475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5c7c0-6378-4ac2-9588-199202eba622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:44.806169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5c7c0-6378-4ac2-9588-199202eba622 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:44.806452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5c7c0-6378-4ac2-9588-199202eba622 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:44.806729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5c7c0-6378-4ac2-9588-199202eba622 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:44.807455 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5c7c0-6378-4ac2-9588-199202eba622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:44.807765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5c7c0-6378-4ac2-9588-199202eba622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:44.810028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5c7c0-6378-4ac2-9588-199202eba622 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:44.822243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1269/2543] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:03:44.826847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104df2a1-82cc-4ddb-8f4f-ea24d51a9194 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:44.839471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104df2a1-82cc-4ddb-8f4f-ea24d51a9194 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:44.844344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72cec6-78d6-4ca0-a360-83f6032e123a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:44.845006 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72cec6-78d6-4ca0-a360-83f6032e123a admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:44.845314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72cec6-78d6-4ca0-a360-83f6032e123a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:44.845590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72cec6-78d6-4ca0-a360-83f6032e123a admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:44.846338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72cec6-78d6-4ca0-a360-83f6032e123a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:44.848267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72cec6-78d6-4ca0-a360-83f6032e123a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:44.976768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1275/2544] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:44 2022] POST /identity/v3/projects => generated 369 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:44.981896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5c7c0-6378-4ac2-9588-199202eba622 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:44.992803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5c7c0-6378-4ac2-9588-199202eba622 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:45.054082 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9180c6-2d94-4523-af0c-0bfe2c761950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:45.054776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9180c6-2d94-4523-af0c-0bfe2c761950 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:45.055067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9180c6-2d94-4523-af0c-0bfe2c761950 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:45.055347 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9180c6-2d94-4523-af0c-0bfe2c761950 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:45.056131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9180c6-2d94-4523-af0c-0bfe2c761950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:45.058131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9180c6-2d94-4523-af0c-0bfe2c761950 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:45.151373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:45.161514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1270/2545] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:44 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:45.162727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72cec6-78d6-4ca0-a360-83f6032e123a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:45.174149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72cec6-78d6-4ca0-a360-83f6032e123a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:45.178962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220563d-b029-47e3-80c1-443b5bf506e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:45.179641 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220563d-b029-47e3-80c1-443b5bf506e5 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:45.179928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220563d-b029-47e3-80c1-443b5bf506e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:45.180209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220563d-b029-47e3-80c1-443b5bf506e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:45.180971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220563d-b029-47e3-80c1-443b5bf506e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:45.182613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220563d-b029-47e3-80c1-443b5bf506e5 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:45.195550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1276/2546] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:45.199698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9180c6-2d94-4523-af0c-0bfe2c761950 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:45.209987 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9180c6-2d94-4523-af0c-0bfe2c761950 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:45.215264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef551c65-3b08-4654-aec4-1aca82814894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:45.215927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef551c65-3b08-4654-aec4-1aca82814894 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:45.216213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef551c65-3b08-4654-aec4-1aca82814894 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:45.216502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef551c65-3b08-4654-aec4-1aca82814894 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:45.217215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef551c65-3b08-4654-aec4-1aca82814894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:45.218911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef551c65-3b08-4654-aec4-1aca82814894 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:45.232477 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1271/2547] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:45.235594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220563d-b029-47e3-80c1-443b5bf506e5 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:45.246201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220563d-b029-47e3-80c1-443b5bf506e5 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:45.251155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b19c3-e274-42ef-ae48-985231b3aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:45.251796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b19c3-e274-42ef-ae48-985231b3aa53 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:45.252082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b19c3-e274-42ef-ae48-985231b3aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:45.252422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b19c3-e274-42ef-ae48-985231b3aa53 admin admin] PATH_INFO: `/v3/projects/99b7dddc880542c28ac15d37eaea5fa8/users/db6f76cc449b4708bae9b59a944568ba/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:45.268166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b19c3-e274-42ef-ae48-985231b3aa53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b7dddc880542c28ac15d37eaea5fa8, user_id=db6f76cc449b4708bae9b59a944568ba, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:45.270955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b19c3-e274-42ef-ae48-985231b3aa53 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:45.353702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1277/2548] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:45 2022] PUT /identity/v3/projects/99b7dddc880542c28ac15d37eaea5fa8/users/db6f76cc449b4708bae9b59a944568ba/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:45.363881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e064e-0b20-4474-a98f-18f98080955c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:45.364361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e064e-0b20-4474-a98f-18f98080955c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:45.364679 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e064e-0b20-4474-a98f-18f98080955c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:45.391663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-230e064e-0b20-4474-a98f-18f98080955c None None] MFA Rules not processed for user `db6f76cc449b4708bae9b59a944568ba`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:45.392885 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-230e064e-0b20-4474-a98f-18f98080955c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:45.459782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1272/2549] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:45.464352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b19c3-e274-42ef-ae48-985231b3aa53 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:45.532694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b19c3-e274-42ef-ae48-985231b3aa53 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:45.537943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e6790-3f21-497b-a78a-3ee436689eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:45.538530 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e6790-3f21-497b-a78a-3ee436689eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:45.538814 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e6790-3f21-497b-a78a-3ee436689eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:45.539099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e6790-3f21-497b-a78a-3ee436689eb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:45.539818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e6790-3f21-497b-a78a-3ee436689eb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:45.541780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e6790-3f21-497b-a78a-3ee436689eb4 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:45.552125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:45.738897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1278/2550] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:45 2022] POST /identity/v3/projects => generated 367 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:45.743058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e064e-0b20-4474-a98f-18f98080955c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:45.754848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e064e-0b20-4474-a98f-18f98080955c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:45.813259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c514b-cbe1-4d5c-93e2-d0d9406360fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:45.813954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c514b-cbe1-4d5c-93e2-d0d9406360fe admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:45.814244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c514b-cbe1-4d5c-93e2-d0d9406360fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:45.814523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c514b-cbe1-4d5c-93e2-d0d9406360fe admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:45.815261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c514b-cbe1-4d5c-93e2-d0d9406360fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:45.817299 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c514b-cbe1-4d5c-93e2-d0d9406360fe admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:45.981307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1273/2551] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:45 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:45.985222 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e6790-3f21-497b-a78a-3ee436689eb4 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:45.995672 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e6790-3f21-497b-a78a-3ee436689eb4 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:46.000496 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ed7be-8924-4b7b-b086-984d819ecca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:46.001164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ed7be-8924-4b7b-b086-984d819ecca6 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:46.001452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ed7be-8924-4b7b-b086-984d819ecca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:46.001728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ed7be-8924-4b7b-b086-984d819ecca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:46.002505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ed7be-8924-4b7b-b086-984d819ecca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:46.004304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ed7be-8924-4b7b-b086-984d819ecca6 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:46.015329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1279/2552] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:46.023895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c514b-cbe1-4d5c-93e2-d0d9406360fe admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:46.034612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c514b-cbe1-4d5c-93e2-d0d9406360fe admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:46.039653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafafcc0-c0cc-4d07-8b37-0793a9d92443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:46.040550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafafcc0-c0cc-4d07-8b37-0793a9d92443 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:46.040876 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafafcc0-c0cc-4d07-8b37-0793a9d92443 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:46.041114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafafcc0-c0cc-4d07-8b37-0793a9d92443 admin admin] PATH_INFO: `/v3/projects/440bf97d2edf4ed1a157a8c1bb33a4bf/users/74345966731d41e99cc603478041ca36/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:46.057940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafafcc0-c0cc-4d07-8b37-0793a9d92443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=440bf97d2edf4ed1a157a8c1bb33a4bf, user_id=74345966731d41e99cc603478041ca36, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:46.060719 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafafcc0-c0cc-4d07-8b37-0793a9d92443 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:46.183896 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ed7be-8924-4b7b-b086-984d819ecca6 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:46.187172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-713ed7be-8924-4b7b-b086-984d819ecca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:46.188626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1274/2553] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:46 2022] PUT /identity/v3/projects/440bf97d2edf4ed1a157a8c1bb33a4bf/users/74345966731d41e99cc603478041ca36/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:46.192514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafafcc0-c0cc-4d07-8b37-0793a9d92443 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:46.199349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ed7be-8924-4b7b-b086-984d819ecca6 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:46.241852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:46.256058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:46.257048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:46.257335 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:46.257749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] PATH_INFO: `/v3/users/ba343822377b4993b0615d29ac304d86` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:46.260281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba343822377b4993b0615d29ac304d86)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:46.271260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafafcc0-c0cc-4d07-8b37-0793a9d92443 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:46.271398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:46.271506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a15ec-292f-4547-ab34-7d9b9937a58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:46.271506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a15ec-292f-4547-ab34-7d9b9937a58c admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:46.271506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a15ec-292f-4547-ab34-7d9b9937a58c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:46.271506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a15ec-292f-4547-ab34-7d9b9937a58c admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:46.271849 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a15ec-292f-4547-ab34-7d9b9937a58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:46.271849 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a15ec-292f-4547-ab34-7d9b9937a58c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:46.282816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1275/2554] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:46.283900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a15ec-292f-4547-ab34-7d9b9937a58c admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:46.294097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a15ec-292f-4547-ab34-7d9b9937a58c admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:46.299066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3adb7f-4148-4253-9f15-d62eb0965956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:46.305287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3adb7f-4148-4253-9f15-d62eb0965956 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:46.305287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3adb7f-4148-4253-9f15-d62eb0965956 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:46.305287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3adb7f-4148-4253-9f15-d62eb0965956 admin admin] PATH_INFO: `/v3/projects/440bf97d2edf4ed1a157a8c1bb33a4bf/users/74345966731d41e99cc603478041ca36/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:46.305287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3adb7f-4148-4253-9f15-d62eb0965956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=440bf97d2edf4ed1a157a8c1bb33a4bf, user_id=74345966731d41e99cc603478041ca36, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:46.305287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3adb7f-4148-4253-9f15-d62eb0965956 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:46.315569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1276/2555] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:46 2022] PUT /identity/v3/projects/440bf97d2edf4ed1a157a8c1bb33a4bf/users/74345966731d41e99cc603478041ca36/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:46.327445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9c356-5738-436a-b2fd-9e146c585265 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:46.327740 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9c356-5738-436a-b2fd-9e146c585265 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:46.328111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9c356-5738-436a-b2fd-9e146c585265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:46.354797 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-4ba9c356-5738-436a-b2fd-9e146c585265 None None] MFA Rules not processed for user `74345966731d41e99cc603478041ca36`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:46.356027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4ba9c356-5738-436a-b2fd-9e146c585265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:46.431773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1277/2556] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:46.443294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9c356-5738-436a-b2fd-9e146c585265 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:46.444744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4ba9c356-5738-436a-b2fd-9e146c585265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:46.484480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba343822377b4993b0615d29ac304d86'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:46.503945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9c356-5738-436a-b2fd-9e146c585265 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:46.508509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93ddba-dd32-41d3-93b0-200a5ed348f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:46.509060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93ddba-dd32-41d3-93b0-200a5ed348f4 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:46.509351 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93ddba-dd32-41d3-93b0-200a5ed348f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:46.509630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93ddba-dd32-41d3-93b0-200a5ed348f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:46.520501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93ddba-dd32-41d3-93b0-200a5ed348f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:46.522031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c93ddba-dd32-41d3-93b0-200a5ed348f4 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:46.532801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1278/2557] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:03:46.591993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba343822377b4993b0615d29ac304d86'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:46.592665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba343822377b4993b0615d29ac304d86'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:46.603146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba343822377b4993b0615d29ac304d86'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:46.604303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba343822377b4993b0615d29ac304d86'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:46.629447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1280/2558] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:46 2022] DELETE /identity/v3/users/ba343822377b4993b0615d29ac304d86 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:46.634831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e05a8-e53d-4f48-8e77-17985ca6bf11 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:46.634831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e05a8-e53d-4f48-8e77-17985ca6bf11 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:46.634831 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e05a8-e53d-4f48-8e77-17985ca6bf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:46.652397 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-7d5e05a8-e53d-4f48-8e77-17985ca6bf11 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:46.653644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7d5e05a8-e53d-4f48-8e77-17985ca6bf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:46.713387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1279/2559] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:46.723779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:46.725226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:46.787221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70645171-6320-4ac6-8968-c7043628776a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:46.790873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db348777-045e-4888-9e56-873129ecccfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:46.791522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db348777-045e-4888-9e56-873129ecccfa service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:46.791813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db348777-045e-4888-9e56-873129ecccfa service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:46.792092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db348777-045e-4888-9e56-873129ecccfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:46.802251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db348777-045e-4888-9e56-873129ecccfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:46.803690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db348777-045e-4888-9e56-873129ecccfa service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:46.810263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:46.816400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1281/2560] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:46.826681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:47.302559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e05a8-e53d-4f48-8e77-17985ca6bf11 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:47.304137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7d5e05a8-e53d-4f48-8e77-17985ca6bf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:47.316865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e05a8-e53d-4f48-8e77-17985ca6bf11 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:47.321884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:47.322816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:47.323101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:47.323409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] PATH_INFO: `/v3/projects/0885fa6efe234605a49ef515fb51bc80` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:47.324666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0885fa6efe234605a49ef515fb51bc80)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:47.326659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:47.485187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0885fa6efe234605a49ef515fb51bc80', 'request_id': 'req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:47.490762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0885fa6efe234605a49ef515fb51bc80', 'request_id': 'req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:47.615011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0885fa6efe234605a49ef515fb51bc80', 'request_id': 'req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:47.627091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] The token cache is being invalidate because project 0885fa6efe234605a49ef515fb51bc80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:47.627385 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:47.628019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:47.629567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1280/2561] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:47 2022] DELETE /identity/v3/projects/0885fa6efe234605a49ef515fb51bc80 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:47.633687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db348777-045e-4888-9e56-873129ecccfa service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:47.644067 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db348777-045e-4888-9e56-873129ecccfa service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:47.700803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:47.701682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:47.702007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:47.702295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] PATH_INFO: `/v3/users/ed5fc043b4bd4a2bbd5e040bb0e80795` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:47.703479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed5fc043b4bd4a2bbd5e040bb0e80795)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:47.706089 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:48.019974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed5fc043b4bd4a2bbd5e040bb0e80795'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.023816 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:48.025256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:48.041922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ae2b3-627f-45a8-8d91-edd79ca67e2a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:48.051103 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:48.052050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:48.052345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:48.052608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] PATH_INFO: `/v3/users/73c370b8968f40618fbac9b3e010b39d` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:48.053847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c370b8968f40618fbac9b3e010b39d)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:48.055857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:48.152409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed5fc043b4bd4a2bbd5e040bb0e80795'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.152959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed5fc043b4bd4a2bbd5e040bb0e80795'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.160627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed5fc043b4bd4a2bbd5e040bb0e80795'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.161366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-11e34c11-7493-4511-9250-f9b5ad5df69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed5fc043b4bd4a2bbd5e040bb0e80795'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.179456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1282/2562] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:47 2022] DELETE /identity/v3/users/ed5fc043b4bd4a2bbd5e040bb0e80795 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:48.302443 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c370b8968f40618fbac9b3e010b39d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.426531 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c370b8968f40618fbac9b3e010b39d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.427173 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c370b8968f40618fbac9b3e010b39d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.439562 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c370b8968f40618fbac9b3e010b39d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.440079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c370b8968f40618fbac9b3e010b39d'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:48.465925 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1281/2563] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:48 2022] DELETE /identity/v3/users/73c370b8968f40618fbac9b3e010b39d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:48.470815 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d7ef-84f9-4f29-bb21-d59ee549a0d7 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:48.473851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d7ef-84f9-4f29-bb21-d59ee549a0d7 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:48.473851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9d7ef-84f9-4f29-bb21-d59ee549a0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:48.489076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-2ef9d7ef-84f9-4f29-bb21-d59ee549a0d7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:48.490355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2ef9d7ef-84f9-4f29-bb21-d59ee549a0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:48.568177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1283/2564] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:48.583484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:48.583484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:48.637640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:48.641523 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e4103-5d0a-43b7-ae6c-c8b34525f40f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:48.645108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5c31e-9553-42c0-904d-57b55d4fa6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:48.645759 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5c31e-9553-42c0-904d-57b55d4fa6bc service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:48.646080 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5c31e-9553-42c0-904d-57b55d4fa6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:48.646353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5c31e-9553-42c0-904d-57b55d4fa6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:48.655576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5c31e-9553-42c0-904d-57b55d4fa6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:48.656949 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5c31e-9553-42c0-904d-57b55d4fa6bc service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:48.666354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1282/2565] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:03:48.689043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d7ef-84f9-4f29-bb21-d59ee549a0d7 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:48.690591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-2ef9d7ef-84f9-4f29-bb21-d59ee549a0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:48.705781 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9d7ef-84f9-4f29-bb21-d59ee549a0d7 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:48.712411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:48.715062 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:48.716154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:48.717049 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] PATH_INFO: `/v3/projects/a871a6b0b25c4ce3a25e9620dbcf6366` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:48.718651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a871a6b0b25c4ce3a25e9620dbcf6366)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:48.720843 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:48.750341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:49.008860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a871a6b0b25c4ce3a25e9620dbcf6366', 'request_id': 'req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.015763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a871a6b0b25c4ce3a25e9620dbcf6366', 'request_id': 'req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.134433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a871a6b0b25c4ce3a25e9620dbcf6366', 'request_id': 'req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.147621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] The token cache is being invalidate because project a871a6b0b25c4ce3a25e9620dbcf6366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:49.148032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.149035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ea9bd23c-6b6c-4e77-9b0b-efd8175bb004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.150764 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1284/2566] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:48 2022] DELETE /identity/v3/projects/a871a6b0b25c4ce3a25e9620dbcf6366 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:49.156218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7584f5d-3596-47d4-bbeb-6e3d4e1913f5 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.156765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7584f5d-3596-47d4-bbeb-6e3d4e1913f5 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.156877 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7584f5d-3596-47d4-bbeb-6e3d4e1913f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.175732 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-d7584f5d-3596-47d4-bbeb-6e3d4e1913f5 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:49.176961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d7584f5d-3596-47d4-bbeb-6e3d4e1913f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:49.241990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1283/2567] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:49.249707 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17198c4a-55a9-4194-862d-751880040238 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.250257 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17198c4a-55a9-4194-862d-751880040238 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.250458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17198c4a-55a9-4194-862d-751880040238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.269214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-17198c4a-55a9-4194-862d-751880040238 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:49.270471 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-17198c4a-55a9-4194-862d-751880040238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:49.279447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1285/2568] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:49.283631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7584f5d-3596-47d4-bbeb-6e3d4e1913f5 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:49.293848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7584f5d-3596-47d4-bbeb-6e3d4e1913f5 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:49.303028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc533e1-d8be-4d24-b371-5841940ecd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:49.303028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc533e1-d8be-4d24-b371-5841940ecd34 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.303028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc533e1-d8be-4d24-b371-5841940ecd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.303028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc533e1-d8be-4d24-b371-5841940ecd34 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.303418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc533e1-d8be-4d24-b371-5841940ecd34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:49.303418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc533e1-d8be-4d24-b371-5841940ecd34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:49.303993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc533e1-d8be-4d24-b371-5841940ecd34 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:49.315643 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17198c4a-55a9-4194-862d-751880040238 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:49.318214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-17198c4a-55a9-4194-862d-751880040238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:49.321904 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1284/2569] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:03:49.329264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc533e1-d8be-4d24-b371-5841940ecd34 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:49.340599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc533e1-d8be-4d24-b371-5841940ecd34 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:49.344236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17198c4a-55a9-4194-862d-751880040238 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:49.346238 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fdce0-56b4-4678-940b-355501f1b93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:49.347074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fdce0-56b4-4678-940b-355501f1b93f admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.347370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fdce0-56b4-4678-940b-355501f1b93f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.347654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fdce0-56b4-4678-940b-355501f1b93f admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.348391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fdce0-56b4-4678-940b-355501f1b93f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:49.349280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:49.350251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.350535 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.350674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fdce0-56b4-4678-940b-355501f1b93f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:49.350800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] PATH_INFO: `/v3/projects/b3099836957a40e793fe9a561204e9b3` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.351992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3099836957a40e793fe9a561204e9b3)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:49.353974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:49.468139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1285/2570] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:49 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:49.472710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fdce0-56b4-4678-940b-355501f1b93f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:49.508990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3099836957a40e793fe9a561204e9b3', 'request_id': 'req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.521904 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3099836957a40e793fe9a561204e9b3', 'request_id': 'req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.543149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fdce0-56b4-4678-940b-355501f1b93f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:49.608916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d53ad7-a800-46ec-9138-9218cc814df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:49.609697 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d53ad7-a800-46ec-9138-9218cc814df5 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.609989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d53ad7-a800-46ec-9138-9218cc814df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.610270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d53ad7-a800-46ec-9138-9218cc814df5 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.611107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d53ad7-a800-46ec-9138-9218cc814df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:49.615120 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d53ad7-a800-46ec-9138-9218cc814df5 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:49.623146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:49.676411 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3099836957a40e793fe9a561204e9b3', 'request_id': 'req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.689263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] The token cache is being invalidate because project b3099836957a40e793fe9a561204e9b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:49.689925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.690468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-c4e161e0-8b24-4a07-8d3e-a4c5c64caf62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:49.692118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1286/2571] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:49 2022] DELETE /identity/v3/projects/b3099836957a40e793fe9a561204e9b3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:49.697015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc76e5-ca74-497c-a9b2-20b607b3c467 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.697304 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc76e5-ca74-497c-a9b2-20b607b3c467 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.697581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc76e5-ca74-497c-a9b2-20b607b3c467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.715932 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c3cc76e5-ca74-497c-a9b2-20b607b3c467 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:49.717745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c3cc76e5-ca74-497c-a9b2-20b607b3c467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:49.726465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1287/2572] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:49.734102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bec5a-07e3-45f5-aed9-7c339f4e2785 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.734388 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bec5a-07e3-45f5-aed9-7c339f4e2785 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.734664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bec5a-07e3-45f5-aed9-7c339f4e2785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.753671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0f5bec5a-07e3-45f5-aed9-7c339f4e2785 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:49.754886 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0f5bec5a-07e3-45f5-aed9-7c339f4e2785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:49.761963 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1286/2573] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:49 2022] POST /identity/v3/users => generated 290 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:49.764345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d53ad7-a800-46ec-9138-9218cc814df5 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:49.764455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1288/2574] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:49.764587 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-59d53ad7-a800-46ec-9138-9218cc814df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:49.767697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bec5a-07e3-45f5-aed9-7c339f4e2785 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:49.776476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d53ad7-a800-46ec-9138-9218cc814df5 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:49.777599 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bec5a-07e3-45f5-aed9-7c339f4e2785 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:49.781851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd220f-fc0b-4cc6-a0ea-ca82b59b9fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:49.782468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768436c3-79ba-43d5-85c4-858babaa801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:49.782603 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd220f-fc0b-4cc6-a0ea-ca82b59b9fdd admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.782835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd220f-fc0b-4cc6-a0ea-ca82b59b9fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.783205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768436c3-79ba-43d5-85c4-858babaa801b admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.783324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd220f-fc0b-4cc6-a0ea-ca82b59b9fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.783426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768436c3-79ba-43d5-85c4-858babaa801b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.783709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768436c3-79ba-43d5-85c4-858babaa801b admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.783942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd220f-fc0b-4cc6-a0ea-ca82b59b9fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:49.784514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768436c3-79ba-43d5-85c4-858babaa801b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:49.784820 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768436c3-79ba-43d5-85c4-858babaa801b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:49.785536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd220f-fc0b-4cc6-a0ea-ca82b59b9fdd admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:49.786409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768436c3-79ba-43d5-85c4-858babaa801b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:49.800419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1287/2575] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:49.807243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd220f-fc0b-4cc6-a0ea-ca82b59b9fdd admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:49.807509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1289/2576] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:03:49.810247 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768436c3-79ba-43d5-85c4-858babaa801b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:49.821075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768436c3-79ba-43d5-85c4-858babaa801b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:49.824429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd220f-fc0b-4cc6-a0ea-ca82b59b9fdd admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:49.826914 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804610c5-1d2c-41e1-8a7e-8cc1baf26a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:49.827595 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804610c5-1d2c-41e1-8a7e-8cc1baf26a51 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.827872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804610c5-1d2c-41e1-8a7e-8cc1baf26a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.828195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804610c5-1d2c-41e1-8a7e-8cc1baf26a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.829070 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804610c5-1d2c-41e1-8a7e-8cc1baf26a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:49.829503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ca948-44a2-4721-99d8-2696416bfbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:49.830286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca948-44a2-4721-99d8-2696416bfbac admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.830641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca948-44a2-4721-99d8-2696416bfbac admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.830851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804610c5-1d2c-41e1-8a7e-8cc1baf26a51 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:49.831034 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca948-44a2-4721-99d8-2696416bfbac admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.831835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ca948-44a2-4721-99d8-2696416bfbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:49.833552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ca948-44a2-4721-99d8-2696416bfbac admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:49.849313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1288/2577] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:49.850773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ca948-44a2-4721-99d8-2696416bfbac admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:49.860917 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ca948-44a2-4721-99d8-2696416bfbac admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:49.865842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ad76d-9d19-4086-9580-2c22fb11c914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:49.866822 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ad76d-9d19-4086-9580-2c22fb11c914 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:49.867102 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ad76d-9d19-4086-9580-2c22fb11c914 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:49.867374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ad76d-9d19-4086-9580-2c22fb11c914 admin admin] PATH_INFO: `/v3/projects/634b7ca060fa457b9660f5a5680c27b6/users/1513d4077dd24a61965219535270f518/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:49.883680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ad76d-9d19-4086-9580-2c22fb11c914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=634b7ca060fa457b9660f5a5680c27b6, user_id=1513d4077dd24a61965219535270f518, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:49.886946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ad76d-9d19-4086-9580-2c22fb11c914 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:49.977838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1290/2578] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:49 2022] POST /identity/v3/projects => generated 403 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:49.982136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804610c5-1d2c-41e1-8a7e-8cc1baf26a51 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.014331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1289/2579] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:49 2022] PUT /identity/v3/projects/634b7ca060fa457b9660f5a5680c27b6/users/1513d4077dd24a61965219535270f518/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:50.020488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b8122-c771-4275-a7ce-9871166bb4cc None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.020488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b8122-c771-4275-a7ce-9871166bb4cc None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.020488 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b8122-c771-4275-a7ce-9871166bb4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.026360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:50.055105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ec8b8122-c771-4275-a7ce-9871166bb4cc None None] MFA Rules not processed for user `1513d4077dd24a61965219535270f518`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:50.056689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ec8b8122-c771-4275-a7ce-9871166bb4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:50.065109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804610c5-1d2c-41e1-8a7e-8cc1baf26a51 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.118549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6046ae-cb2d-4b80-bc77-24d90a1ef3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.119232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6046ae-cb2d-4b80-bc77-24d90a1ef3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.119513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6046ae-cb2d-4b80-bc77-24d90a1ef3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.119789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6046ae-cb2d-4b80-bc77-24d90a1ef3f8 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.120557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6046ae-cb2d-4b80-bc77-24d90a1ef3f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.122682 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6046ae-cb2d-4b80-bc77-24d90a1ef3f8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.129621 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1290/2580] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:50.134692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b8122-c771-4275-a7ce-9871166bb4cc None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.146046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b8122-c771-4275-a7ce-9871166bb4cc None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.150769 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4a31c-4927-4d99-beaf-1d676a209a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.151450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4a31c-4927-4d99-beaf-1d676a209a29 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.154243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4a31c-4927-4d99-beaf-1d676a209a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.154243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4a31c-4927-4d99-beaf-1d676a209a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.154243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4a31c-4927-4d99-beaf-1d676a209a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.155344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4a31c-4927-4d99-beaf-1d676a209a29 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.273655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1291/2581] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:49 2022] POST /identity/v3/users => generated 302 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:50.275165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1291/2582] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:50 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:50.277489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6046ae-cb2d-4b80-bc77-24d90a1ef3f8 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.278905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4a31c-4927-4d99-beaf-1d676a209a29 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.287973 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6046ae-cb2d-4b80-bc77-24d90a1ef3f8 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.346691 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4a31c-4927-4d99-beaf-1d676a209a29 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.351631 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc22ea-5aee-411a-997f-bceb33b04b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.352312 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc22ea-5aee-411a-997f-bceb33b04b1b admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.352600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc22ea-5aee-411a-997f-bceb33b04b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.352882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc22ea-5aee-411a-997f-bceb33b04b1b admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.353582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc22ea-5aee-411a-997f-bceb33b04b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.355597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc22ea-5aee-411a-997f-bceb33b04b1b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.356144 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823e2b-09ee-457b-a727-49f7fa24ed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.356944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823e2b-09ee-457b-a727-49f7fa24ed8c admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.357331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823e2b-09ee-457b-a727-49f7fa24ed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.357709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b823e2b-09ee-457b-a727-49f7fa24ed8c admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.358578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b823e2b-09ee-457b-a727-49f7fa24ed8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.360414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b823e2b-09ee-457b-a727-49f7fa24ed8c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.371319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1292/2583] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:50.375173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823e2b-09ee-457b-a727-49f7fa24ed8c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.384921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b823e2b-09ee-457b-a727-49f7fa24ed8c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.390408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12d27-f3f4-4311-b0ea-8b802cb0cc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.391056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12d27-f3f4-4311-b0ea-8b802cb0cc9e admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.391349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12d27-f3f4-4311-b0ea-8b802cb0cc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.391626 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f12d27-f3f4-4311-b0ea-8b802cb0cc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.392341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f12d27-f3f4-4311-b0ea-8b802cb0cc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.393937 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f12d27-f3f4-4311-b0ea-8b802cb0cc9e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.404519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1293/2584] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:50.408609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12d27-f3f4-4311-b0ea-8b802cb0cc9e admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.419536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f12d27-f3f4-4311-b0ea-8b802cb0cc9e admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.425156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c31c0-34e7-45a7-8174-bfe36b4d5d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.426505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c31c0-34e7-45a7-8174-bfe36b4d5d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.426505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c31c0-34e7-45a7-8174-bfe36b4d5d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.426736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c31c0-34e7-45a7-8174-bfe36b4d5d17 admin admin] PATH_INFO: `/v3/projects/b807646ffbc14c0e96acf2cf042ab9b8/users/3795aa6537d04bd58d6f0aa26e4376d4/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.442667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c31c0-34e7-45a7-8174-bfe36b4d5d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b807646ffbc14c0e96acf2cf042ab9b8, user_id=3795aa6537d04bd58d6f0aa26e4376d4, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.445295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c31c0-34e7-45a7-8174-bfe36b4d5d17 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.456939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1292/2585] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:50 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:50.457163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc22ea-5aee-411a-997f-bceb33b04b1b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.458738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-4dfc22ea-5aee-411a-997f-bceb33b04b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:50.468679 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:50.472585 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc22ea-5aee-411a-997f-bceb33b04b1b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.477172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.478067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.478354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.478635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] PATH_INFO: `/v3/users/db6f76cc449b4708bae9b59a944568ba` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.479886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6f76cc449b4708bae9b59a944568ba)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.482357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.581708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1294/2586] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:50 2022] PUT /identity/v3/projects/b807646ffbc14c0e96acf2cf042ab9b8/users/3795aa6537d04bd58d6f0aa26e4376d4/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:50.582591 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c31c0-34e7-45a7-8174-bfe36b4d5d17 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.647087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c31c0-34e7-45a7-8174-bfe36b4d5d17 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.652649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176979e0-c4a8-406d-9f03-b42f5cb00643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.653509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176979e0-c4a8-406d-9f03-b42f5cb00643 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.653794 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176979e0-c4a8-406d-9f03-b42f5cb00643 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.654105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176979e0-c4a8-406d-9f03-b42f5cb00643 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.654836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176979e0-c4a8-406d-9f03-b42f5cb00643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.656437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176979e0-c4a8-406d-9f03-b42f5cb00643 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.667230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1295/2587] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:50.668590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d56e3-bcb8-4f65-aa53-758139c075e5 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.669016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d56e3-bcb8-4f65-aa53-758139c075e5 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.675261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d56e3-bcb8-4f65-aa53-758139c075e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.687483 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-9f1d56e3-bcb8-4f65-aa53-758139c075e5 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:50.688752 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9f1d56e3-bcb8-4f65-aa53-758139c075e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:50.696551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1296/2588] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:50.697484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d56e3-bcb8-4f65-aa53-758139c075e5 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.709010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d56e3-bcb8-4f65-aa53-758139c075e5 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.713653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23940c64-7a03-4c35-9568-a90de76c035a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.714985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23940c64-7a03-4c35-9568-a90de76c035a admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.715269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23940c64-7a03-4c35-9568-a90de76c035a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.715544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23940c64-7a03-4c35-9568-a90de76c035a admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.716301 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23940c64-7a03-4c35-9568-a90de76c035a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.717900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23940c64-7a03-4c35-9568-a90de76c035a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.728095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1297/2589] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:50.728927 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23940c64-7a03-4c35-9568-a90de76c035a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.730681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-23940c64-7a03-4c35-9568-a90de76c035a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:50.764268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6f76cc449b4708bae9b59a944568ba'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:50.792998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23940c64-7a03-4c35-9568-a90de76c035a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.796818 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788c159-d4d2-46b9-adc8-027b4d1005d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.803652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788c159-d4d2-46b9-adc8-027b4d1005d6 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.803652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788c159-d4d2-46b9-adc8-027b4d1005d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.803652 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788c159-d4d2-46b9-adc8-027b4d1005d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.807173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788c159-d4d2-46b9-adc8-027b4d1005d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.808640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788c159-d4d2-46b9-adc8-027b4d1005d6 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.818145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1298/2590] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:50.818915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788c159-d4d2-46b9-adc8-027b4d1005d6 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:50.828509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788c159-d4d2-46b9-adc8-027b4d1005d6 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:50.830911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:50.835947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7984fb4-8e3f-42c9-a86c-c7488a23abcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:50.849797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7984fb4-8e3f-42c9-a86c-c7488a23abcf admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.849797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7984fb4-8e3f-42c9-a86c-c7488a23abcf admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.849797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7984fb4-8e3f-42c9-a86c-c7488a23abcf admin admin] PATH_INFO: `/v3/projects/3ac344bccfad4e6a960a0c99bba2cada/users/74fda6e0f4d84e06b96d20a35e0b5c0f/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.854500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7984fb4-8e3f-42c9-a86c-c7488a23abcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac344bccfad4e6a960a0c99bba2cada, user_id=74fda6e0f4d84e06b96d20a35e0b5c0f, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:50.857156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7984fb4-8e3f-42c9-a86c-c7488a23abcf admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:50.877357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6f76cc449b4708bae9b59a944568ba'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:50.878017 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6f76cc449b4708bae9b59a944568ba'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:50.885782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6f76cc449b4708bae9b59a944568ba'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:50.888097 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-d86febb3-9f34-40d2-9e70-b17e6d0fb6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6f76cc449b4708bae9b59a944568ba'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:50.906479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1293/2591] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:50 2022] DELETE /identity/v3/users/db6f76cc449b4708bae9b59a944568ba => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:50.911033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3713f1f-12a3-4502-bf9e-abe1c5a70739 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:50.911326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3713f1f-12a3-4502-bf9e-abe1c5a70739 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:50.911607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3713f1f-12a3-4502-bf9e-abe1c5a70739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:50.928916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-a3713f1f-12a3-4502-bf9e-abe1c5a70739 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:50.930157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a3713f1f-12a3-4502-bf9e-abe1c5a70739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:50.993522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1294/2592] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:51.003966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3713f1f-12a3-4502-bf9e-abe1c5a70739 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:51.005363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a3713f1f-12a3-4502-bf9e-abe1c5a70739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:51.048870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1299/2593] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:50 2022] PUT /identity/v3/projects/3ac344bccfad4e6a960a0c99bba2cada/users/74fda6e0f4d84e06b96d20a35e0b5c0f/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:51.072016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36808-dd29-4a47-bed4-29b4208254c9 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:51.072385 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36808-dd29-4a47-bed4-29b4208254c9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:51.072580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d36808-dd29-4a47-bed4-29b4208254c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:51.075841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3713f1f-12a3-4502-bf9e-abe1c5a70739 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:51.107449 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-92d36808-dd29-4a47-bed4-29b4208254c9 None None] MFA Rules not processed for user `74fda6e0f4d84e06b96d20a35e0b5c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:51.108940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-92d36808-dd29-4a47-bed4-29b4208254c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:51.116905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:51.154307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa967da-7fa4-4905-9aac-2c4573e287d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:51.154980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa967da-7fa4-4905-9aac-2c4573e287d8 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:51.155274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa967da-7fa4-4905-9aac-2c4573e287d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:51.155553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa967da-7fa4-4905-9aac-2c4573e287d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:51.219419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1300/2594] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:51.226893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d36808-dd29-4a47-bed4-29b4208254c9 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:51.228447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa967da-7fa4-4905-9aac-2c4573e287d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:51.230763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa967da-7fa4-4905-9aac-2c4573e287d8 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:51.235299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-92d36808-dd29-4a47-bed4-29b4208254c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:51.247131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1295/2595] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:03:51.254653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d36808-dd29-4a47-bed4-29b4208254c9 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:51.260428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85120b2-11fa-42d5-a384-168a037aacc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:51.264502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85120b2-11fa-42d5-a384-168a037aacc3 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:51.264502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85120b2-11fa-42d5-a384-168a037aacc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:51.264502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85120b2-11fa-42d5-a384-168a037aacc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:51.264502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85120b2-11fa-42d5-a384-168a037aacc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:51.264870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85120b2-11fa-42d5-a384-168a037aacc3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:51.281003 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:51.460500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1301/2596] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:51 2022] POST /identity/v3/projects => generated 377 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:51.465142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa967da-7fa4-4905-9aac-2c4573e287d8 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:51.475023 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa967da-7fa4-4905-9aac-2c4573e287d8 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:51.536652 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f361e-c8a2-4594-9c0c-203eef748232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:51.537334 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f361e-c8a2-4594-9c0c-203eef748232 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:51.537616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f361e-c8a2-4594-9c0c-203eef748232 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:51.537920 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f361e-c8a2-4594-9c0c-203eef748232 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:51.538706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f361e-c8a2-4594-9c0c-203eef748232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:51.540565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f361e-c8a2-4594-9c0c-203eef748232 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:51.660921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1296/2597] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:51 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:51.664669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85120b2-11fa-42d5-a384-168a037aacc3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:51.675734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85120b2-11fa-42d5-a384-168a037aacc3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:51.680852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10441a8d-80c6-4693-8188-e439ca2a85fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:51.686758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441a8d-80c6-4693-8188-e439ca2a85fe admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:51.686758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441a8d-80c6-4693-8188-e439ca2a85fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:51.686758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441a8d-80c6-4693-8188-e439ca2a85fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:51.686758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10441a8d-80c6-4693-8188-e439ca2a85fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:51.686758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10441a8d-80c6-4693-8188-e439ca2a85fe admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:51.696110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1302/2598] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:51.701037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f361e-c8a2-4594-9c0c-203eef748232 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:51.712106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f361e-c8a2-4594-9c0c-203eef748232 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:51.716968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898fd64-5647-45ba-9bc3-0c9b8063023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:51.721129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898fd64-5647-45ba-9bc3-0c9b8063023f admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:51.721129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898fd64-5647-45ba-9bc3-0c9b8063023f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:51.721129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898fd64-5647-45ba-9bc3-0c9b8063023f admin admin] PATH_INFO: `/v3/projects/4592f3be4eab4323979f321efa1aaff1/users/5b72aa635501455bac35a97646e047c8/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:51.735673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898fd64-5647-45ba-9bc3-0c9b8063023f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4592f3be4eab4323979f321efa1aaff1, user_id=5b72aa635501455bac35a97646e047c8, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:51.738670 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898fd64-5647-45ba-9bc3-0c9b8063023f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:51.844610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1297/2599] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:51 2022] PUT /identity/v3/projects/4592f3be4eab4323979f321efa1aaff1/users/5b72aa635501455bac35a97646e047c8/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:51.848634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10441a8d-80c6-4693-8188-e439ca2a85fe admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:51.859419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10441a8d-80c6-4693-8188-e439ca2a85fe admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:51.875314 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898fd64-5647-45ba-9bc3-0c9b8063023f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:51.878018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0898fd64-5647-45ba-9bc3-0c9b8063023f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:51.940950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa338ab3-114e-4b8d-a741-57eec02c99ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:51.941745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa338ab3-114e-4b8d-a741-57eec02c99ce admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:51.942173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa338ab3-114e-4b8d-a741-57eec02c99ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:51.942522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa338ab3-114e-4b8d-a741-57eec02c99ce admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:51.943320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa338ab3-114e-4b8d-a741-57eec02c99ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:51.944976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa338ab3-114e-4b8d-a741-57eec02c99ce admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:51.956555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898fd64-5647-45ba-9bc3-0c9b8063023f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:51.957922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1303/2600] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:51.962129 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:51.962509 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa338ab3-114e-4b8d-a741-57eec02c99ce admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:51.963289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:51.963289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:51.963609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] PATH_INFO: `/v3/projects/99b7dddc880542c28ac15d37eaea5fa8` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:51.965647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b7dddc880542c28ac15d37eaea5fa8)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:51.968265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:51.975052 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa338ab3-114e-4b8d-a741-57eec02c99ce admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:51.982577 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f2bdd-1add-400b-83c6-85758d186cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:51.983476 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f2bdd-1add-400b-83c6-85758d186cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:51.983762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f2bdd-1add-400b-83c6-85758d186cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:51.984044 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f2bdd-1add-400b-83c6-85758d186cd9 admin admin] PATH_INFO: `/v3/projects/4592f3be4eab4323979f321efa1aaff1/users/5b72aa635501455bac35a97646e047c8/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:51.986550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f2bdd-1add-400b-83c6-85758d186cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4592f3be4eab4323979f321efa1aaff1, user_id=5b72aa635501455bac35a97646e047c8, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:51.989196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f2bdd-1add-400b-83c6-85758d186cd9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:52.000771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1304/2601] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:51 2022] PUT /identity/v3/projects/4592f3be4eab4323979f321efa1aaff1/users/5b72aa635501455bac35a97646e047c8/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:52.011377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ea6e0-1dd5-4e38-93f0-c69f7c313946 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:52.011671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ea6e0-1dd5-4e38-93f0-c69f7c313946 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:52.012039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ea6e0-1dd5-4e38-93f0-c69f7c313946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:52.041687 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-fd8ea6e0-1dd5-4e38-93f0-c69f7c313946 None None] MFA Rules not processed for user `5b72aa635501455bac35a97646e047c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:52.043538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fd8ea6e0-1dd5-4e38-93f0-c69f7c313946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:52.117251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1305/2602] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:52.125896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b7dddc880542c28ac15d37eaea5fa8', 'request_id': 'req-b2794916-a52a-4022-928a-e08e7a295e93'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.129240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ea6e0-1dd5-4e38-93f0-c69f7c313946 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:52.130701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fd8ea6e0-1dd5-4e38-93f0-c69f7c313946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:52.133774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b7dddc880542c28ac15d37eaea5fa8', 'request_id': 'req-b2794916-a52a-4022-928a-e08e7a295e93'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.175021 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:52.191743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ea6e0-1dd5-4e38-93f0-c69f7c313946 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:52.195375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:52.208615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b7dddc880542c28ac15d37eaea5fa8', 'request_id': 'req-b2794916-a52a-4022-928a-e08e7a295e93'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.221291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] The token cache is being invalidate because project 99b7dddc880542c28ac15d37eaea5fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:52.221582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.222309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.223894 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1298/2603] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:51 2022] DELETE /identity/v3/projects/99b7dddc880542c28ac15d37eaea5fa8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:52.227793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:52.229125 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:52.243110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee85fdf-4bf8-487d-94f4-86e969e01fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:52.243917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee85fdf-4bf8-487d-94f4-86e969e01fec service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:52.244204 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee85fdf-4bf8-487d-94f4-86e969e01fec service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:52.244519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee85fdf-4bf8-487d-94f4-86e969e01fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:52.246424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-9ee85fdf-4bf8-487d-94f4-86e969e01fec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:52.302537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2794916-a52a-4022-928a-e08e7a295e93 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:52.307675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:52.308644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:52.308921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:52.309198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] PATH_INFO: `/v3/users/74345966731d41e99cc603478041ca36` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:52.310444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74345966731d41e99cc603478041ca36)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:52.312887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:52.314192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee85fdf-4bf8-487d-94f4-86e969e01fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:52.315758 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee85fdf-4bf8-487d-94f4-86e969e01fec service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:52.326614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1306/2604] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:03:52.484678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4cf72-da0b-4cfa-a57d-8362221ab8f7 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:52.485000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4cf72-da0b-4cfa-a57d-8362221ab8f7 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:52.485307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4cf72-da0b-4cfa-a57d-8362221ab8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:52.514029 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d5d4cf72-da0b-4cfa-a57d-8362221ab8f7 None None] MFA Rules not processed for user `3795aa6537d04bd58d6f0aa26e4376d4`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:52.515246 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d5d4cf72-da0b-4cfa-a57d-8362221ab8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:52.585419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1307/2605] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:52.586580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4cf72-da0b-4cfa-a57d-8362221ab8f7 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:52.588048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d5d4cf72-da0b-4cfa-a57d-8362221ab8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:52.595072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74345966731d41e99cc603478041ca36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.601610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4cf72-da0b-4cfa-a57d-8362221ab8f7 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:52.605444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:52.606133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:52.606416 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:52.606690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:52.608603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:52.669716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:52.671207 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:52.673374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74345966731d41e99cc603478041ca36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.674036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74345966731d41e99cc603478041ca36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.682170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1308/2606] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:52.683491 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:52.683753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74345966731d41e99cc603478041ca36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.684588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74345966731d41e99cc603478041ca36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:52.685019 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:52.701710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc6e63-c6b2-4072-802f-6c1ae3d43a3c service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:52.703755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1299/2607] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:52 2022] DELETE /identity/v3/users/74345966731d41e99cc603478041ca36 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:52.707041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31244e6c-63fb-4f9b-9d87-da5892ae02ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:52.708485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31244e6c-63fb-4f9b-9d87-da5892ae02ac admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:52.708575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31244e6c-63fb-4f9b-9d87-da5892ae02ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:52.708800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31244e6c-63fb-4f9b-9d87-da5892ae02ac admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:52.709759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31244e6c-63fb-4f9b-9d87-da5892ae02ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:52.711655 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31244e6c-63fb-4f9b-9d87-da5892ae02ac admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:52.839433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1309/2608] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:52 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:52.843607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:52.845111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:52.914780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5983f6-ae18-4154-819e-11d84acae0d1 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:52.982295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16744-f94d-4216-9abb-4b1d19f6d402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:52.982457 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16744-f94d-4216-9abb-4b1d19f6d402 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:52.982832 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16744-f94d-4216-9abb-4b1d19f6d402 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:52.982961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16744-f94d-4216-9abb-4b1d19f6d402 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:52.983674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d16744-f94d-4216-9abb-4b1d19f6d402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:52.985671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d16744-f94d-4216-9abb-4b1d19f6d402 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:53.097707 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1300/2609] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:52 2022] POST /identity/v3/users => generated 309 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:53.099427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31244e6c-63fb-4f9b-9d87-da5892ae02ac admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:53.110234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31244e6c-63fb-4f9b-9d87-da5892ae02ac admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:53.114767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2545be-32a5-4fdc-b8dd-2c41f26e383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:53.115436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2545be-32a5-4fdc-b8dd-2c41f26e383f admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:53.115726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2545be-32a5-4fdc-b8dd-2c41f26e383f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:53.116061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2545be-32a5-4fdc-b8dd-2c41f26e383f admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:53.116796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2545be-32a5-4fdc-b8dd-2c41f26e383f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:53.118427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2545be-32a5-4fdc-b8dd-2c41f26e383f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:53.130028 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1310/2610] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:53.134620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16744-f94d-4216-9abb-4b1d19f6d402 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:53.137288 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:53.152729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16744-f94d-4216-9abb-4b1d19f6d402 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:53.157782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e34380-089d-4bcc-b6d2-e47ed8e13b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:53.158713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e34380-089d-4bcc-b6d2-e47ed8e13b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:53.158997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e34380-089d-4bcc-b6d2-e47ed8e13b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:53.159275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e34380-089d-4bcc-b6d2-e47ed8e13b57 admin admin] PATH_INFO: `/v3/projects/f16c3e2474fd4e22b9ca883d93a80bc0/users/153d25021aaf4196a839068cf367ee31/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:53.175270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e34380-089d-4bcc-b6d2-e47ed8e13b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16c3e2474fd4e22b9ca883d93a80bc0, user_id=153d25021aaf4196a839068cf367ee31, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:53.177954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e34380-089d-4bcc-b6d2-e47ed8e13b57 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:53.265418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:53.301725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2545be-32a5-4fdc-b8dd-2c41f26e383f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:53.303803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0f2545be-32a5-4fdc-b8dd-2c41f26e383f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:53.303929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1301/2611] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:53 2022] PUT /identity/v3/projects/f16c3e2474fd4e22b9ca883d93a80bc0/users/153d25021aaf4196a839068cf367ee31/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:53.308031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e34380-089d-4bcc-b6d2-e47ed8e13b57 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:53.323943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2545be-32a5-4fdc-b8dd-2c41f26e383f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:53.377179 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e34380-089d-4bcc-b6d2-e47ed8e13b57 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:53.382360 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f10af3-ff0f-4398-9aa5-b4d005fe9cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:53.383204 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10af3-ff0f-4398-9aa5-b4d005fe9cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:53.383491 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10af3-ff0f-4398-9aa5-b4d005fe9cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:53.383775 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10af3-ff0f-4398-9aa5-b4d005fe9cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:53.384604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f10af3-ff0f-4398-9aa5-b4d005fe9cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:53.386282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f10af3-ff0f-4398-9aa5-b4d005fe9cd0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:53.387292 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:53.388229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:53.388512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:53.388785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] PATH_INFO: `/v3/projects/440bf97d2edf4ed1a157a8c1bb33a4bf` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:53.390109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=440bf97d2edf4ed1a157a8c1bb33a4bf)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:53.392081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:53.397177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1302/2612] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:53.402073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f10af3-ff0f-4398-9aa5-b4d005fe9cd0 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:53.412755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f10af3-ff0f-4398-9aa5-b4d005fe9cd0 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:53.418437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b29dcf-4ef8-40fe-ae4a-8a3b567b2d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:53.419613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b29dcf-4ef8-40fe-ae4a-8a3b567b2d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:53.419905 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b29dcf-4ef8-40fe-ae4a-8a3b567b2d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:53.420184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b29dcf-4ef8-40fe-ae4a-8a3b567b2d03 admin admin] PATH_INFO: `/v3/projects/f16c3e2474fd4e22b9ca883d93a80bc0/users/153d25021aaf4196a839068cf367ee31/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:53.422525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b29dcf-4ef8-40fe-ae4a-8a3b567b2d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16c3e2474fd4e22b9ca883d93a80bc0, user_id=153d25021aaf4196a839068cf367ee31, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:53.425224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b29dcf-4ef8-40fe-ae4a-8a3b567b2d03 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:53.436479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1303/2613] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:53 2022] PUT /identity/v3/projects/f16c3e2474fd4e22b9ca883d93a80bc0/users/153d25021aaf4196a839068cf367ee31/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:53.588458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440bf97d2edf4ed1a157a8c1bb33a4bf', 'request_id': 'req-3e3a7a41-139d-4516-a484-55d5f7fd2e64'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:53.599428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440bf97d2edf4ed1a157a8c1bb33a4bf', 'request_id': 'req-3e3a7a41-139d-4516-a484-55d5f7fd2e64'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:53.673286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440bf97d2edf4ed1a157a8c1bb33a4bf', 'request_id': 'req-3e3a7a41-139d-4516-a484-55d5f7fd2e64'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:53.685666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] The token cache is being invalidate because project 440bf97d2edf4ed1a157a8c1bb33a4bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:53.685996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:53.686667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-3e3a7a41-139d-4516-a484-55d5f7fd2e64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:53.688226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1311/2614] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:53 2022] DELETE /identity/v3/projects/440bf97d2edf4ed1a157a8c1bb33a4bf => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:53.693175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a66a23-5620-4a1e-9571-c248991bee26 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:53.693667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a66a23-5620-4a1e-9571-c248991bee26 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:53.693753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a66a23-5620-4a1e-9571-c248991bee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:53.713329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-68a66a23-5620-4a1e-9571-c248991bee26 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:53.715187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-68a66a23-5620-4a1e-9571-c248991bee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:53.787277 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1304/2615] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:53.787645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75837011-fa23-4e51-a63e-f59477c7b03a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:53.787645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75837011-fa23-4e51-a63e-f59477c7b03a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:53.787645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75837011-fa23-4e51-a63e-f59477c7b03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:53.815219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-75837011-fa23-4e51-a63e-f59477c7b03a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:53.816560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-75837011-fa23-4e51-a63e-f59477c7b03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:53.826677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1312/2616] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:53.838482 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a66a23-5620-4a1e-9571-c248991bee26 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:53.849755 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a66a23-5620-4a1e-9571-c248991bee26 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:53.854720 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08abd3e-9cba-4775-a9ee-ddbe7dc696e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:53.855395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08abd3e-9cba-4775-a9ee-ddbe7dc696e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:53.855677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08abd3e-9cba-4775-a9ee-ddbe7dc696e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:53.855952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08abd3e-9cba-4775-a9ee-ddbe7dc696e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:53.856851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08abd3e-9cba-4775-a9ee-ddbe7dc696e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:53.857161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08abd3e-9cba-4775-a9ee-ddbe7dc696e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:53.859013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08abd3e-9cba-4775-a9ee-ddbe7dc696e0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:53.872156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1305/2617] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:03:53.876562 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75837011-fa23-4e51-a63e-f59477c7b03a None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:53.886923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75837011-fa23-4e51-a63e-f59477c7b03a None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:53.902757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1192a08-151b-4b35-b17f-af7e3ae8df77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:53.902757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1192a08-151b-4b35-b17f-af7e3ae8df77 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:53.902757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1192a08-151b-4b35-b17f-af7e3ae8df77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:53.902757 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1192a08-151b-4b35-b17f-af7e3ae8df77 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:53.903172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1192a08-151b-4b35-b17f-af7e3ae8df77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:53.903172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1192a08-151b-4b35-b17f-af7e3ae8df77 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:54.095541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1313/2618] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:53 2022] POST /identity/v3/projects => generated 385 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:54.096658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08abd3e-9cba-4775-a9ee-ddbe7dc696e0 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:54.109886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08abd3e-9cba-4775-a9ee-ddbe7dc696e0 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:54.173349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f124ff-303d-49f0-970d-5478521aa445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:54.174056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f124ff-303d-49f0-970d-5478521aa445 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:54.174340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f124ff-303d-49f0-970d-5478521aa445 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:54.174611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f124ff-303d-49f0-970d-5478521aa445 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:54.175307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f124ff-303d-49f0-970d-5478521aa445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:54.177218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f124ff-303d-49f0-970d-5478521aa445 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:54.415105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1306/2619] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:54 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:03:54.419307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1192a08-151b-4b35-b17f-af7e3ae8df77 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:54.430540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1192a08-151b-4b35-b17f-af7e3ae8df77 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:54.435433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9eeab9-3505-43ce-bb03-f6d21bbea270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:54.436274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9eeab9-3505-43ce-bb03-f6d21bbea270 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:54.436864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9eeab9-3505-43ce-bb03-f6d21bbea270 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:54.436864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9eeab9-3505-43ce-bb03-f6d21bbea270 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:54.437558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9eeab9-3505-43ce-bb03-f6d21bbea270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:54.439177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9eeab9-3505-43ce-bb03-f6d21bbea270 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:54.453558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1314/2620] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:54.455053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f124ff-303d-49f0-970d-5478521aa445 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:54.465598 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f124ff-303d-49f0-970d-5478521aa445 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:54.470261 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b495dd-398d-4d54-8366-b589dcab57d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:54.470945 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b495dd-398d-4d54-8366-b589dcab57d1 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:54.471233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b495dd-398d-4d54-8366-b589dcab57d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:54.471512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b495dd-398d-4d54-8366-b589dcab57d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:54.473077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b495dd-398d-4d54-8366-b589dcab57d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:54.473936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b495dd-398d-4d54-8366-b589dcab57d1 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:54.486543 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1307/2621] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:54.489307 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9eeab9-3505-43ce-bb03-f6d21bbea270 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:54.495276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:54.501176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9eeab9-3505-43ce-bb03-f6d21bbea270 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:54.506221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2d3bb-073d-4c52-b2e4-2ecaa11cb938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:54.507186 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d3bb-073d-4c52-b2e4-2ecaa11cb938 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:54.507525 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d3bb-073d-4c52-b2e4-2ecaa11cb938 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:54.507807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d3bb-073d-4c52-b2e4-2ecaa11cb938 admin admin] PATH_INFO: `/v3/projects/fb6073886a364274be529371945e0055/users/cb06934ec180432fb64e77b01cab1c84/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:54.525199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2d3bb-073d-4c52-b2e4-2ecaa11cb938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6073886a364274be529371945e0055, user_id=cb06934ec180432fb64e77b01cab1c84, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:54.528565 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2d3bb-073d-4c52-b2e4-2ecaa11cb938 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:54.616219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1315/2622] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:54 2022] PUT /identity/v3/projects/fb6073886a364274be529371945e0055/users/cb06934ec180432fb64e77b01cab1c84/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:54.628593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9882a-bf20-46b1-952e-940ef6fc8875 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:54.628882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9882a-bf20-46b1-952e-940ef6fc8875 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:54.629192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9882a-bf20-46b1-952e-940ef6fc8875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:54.658929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-2ac9882a-bf20-46b1-952e-940ef6fc8875 None None] MFA Rules not processed for user `cb06934ec180432fb64e77b01cab1c84`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:54.660226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2ac9882a-bf20-46b1-952e-940ef6fc8875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:54.738756 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1308/2623] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:54.751116 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2d3bb-073d-4c52-b2e4-2ecaa11cb938 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:54.753010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e1d2d3bb-073d-4c52-b2e4-2ecaa11cb938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:54.758851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:54.823693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2d3bb-073d-4c52-b2e4-2ecaa11cb938 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:54.827537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848b2eb-cb0c-49c9-b88d-1d009413602c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:54.828200 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848b2eb-cb0c-49c9-b88d-1d009413602c service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:54.828480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848b2eb-cb0c-49c9-b88d-1d009413602c service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:54.828860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848b2eb-cb0c-49c9-b88d-1d009413602c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:54.838349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848b2eb-cb0c-49c9-b88d-1d009413602c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:54.840298 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848b2eb-cb0c-49c9-b88d-1d009413602c service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:54.850437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1316/2624] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:54.873590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9882a-bf20-46b1-952e-940ef6fc8875 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:54.941572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9882a-bf20-46b1-952e-940ef6fc8875 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:54.946305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:54.947190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:54.947514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:54.947796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] PATH_INFO: `/v3/users/cb06934ec180432fb64e77b01cab1c84` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:54.949042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb06934ec180432fb64e77b01cab1c84)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:54.951067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:55.248157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb06934ec180432fb64e77b01cab1c84'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:55.363415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb06934ec180432fb64e77b01cab1c84'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:55.372521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb06934ec180432fb64e77b01cab1c84'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:55.376114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb06934ec180432fb64e77b01cab1c84'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:55.378225 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-122f9af9-da8e-443b-bbab-a101b983ddc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb06934ec180432fb64e77b01cab1c84'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:55.401217 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab67e40-99f7-41a8-8112-0b3c91861dfa None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:55.402383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab67e40-99f7-41a8-8112-0b3c91861dfa None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:55.402502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab67e40-99f7-41a8-8112-0b3c91861dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:55.417497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1309/2625] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:54 2022] DELETE /identity/v3/users/cb06934ec180432fb64e77b01cab1c84 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:55.424728 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b703e7-9615-40eb-ad64-f92a564a6429 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:55.425149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b703e7-9615-40eb-ad64-f92a564a6429 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:55.425518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b703e7-9615-40eb-ad64-f92a564a6429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:55.434087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cab67e40-99f7-41a8-8112-0b3c91861dfa None None] MFA Rules not processed for user `153d25021aaf4196a839068cf367ee31`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:55.436586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cab67e40-99f7-41a8-8112-0b3c91861dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:55.449040 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-39b703e7-9615-40eb-ad64-f92a564a6429 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:55.450622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-39b703e7-9615-40eb-ad64-f92a564a6429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:55.546162 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1317/2626] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:55 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:55.584689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1310/2627] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:55.604512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab67e40-99f7-41a8-8112-0b3c91861dfa None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:55.604512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cab67e40-99f7-41a8-8112-0b3c91861dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:55.702923 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab67e40-99f7-41a8-8112-0b3c91861dfa None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:55.709455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f56103-5603-4172-bb4a-5a9cdbb945b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:55.710480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f56103-5603-4172-bb4a-5a9cdbb945b2 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:55.711430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f56103-5603-4172-bb4a-5a9cdbb945b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:55.711430 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f56103-5603-4172-bb4a-5a9cdbb945b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:55.722976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f56103-5603-4172-bb4a-5a9cdbb945b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:55.724585 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f56103-5603-4172-bb4a-5a9cdbb945b2 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:55.739664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1318/2628] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:56.131233 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:56.685049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b703e7-9615-40eb-ad64-f92a564a6429 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:56.687010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-39b703e7-9615-40eb-ad64-f92a564a6429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:56.700722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b703e7-9615-40eb-ad64-f92a564a6429 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:56.705509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:56.706426 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:56.706713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:56.707036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] PATH_INFO: `/v3/projects/fb6073886a364274be529371945e0055` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:56.708325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6073886a364274be529371945e0055)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:56.710413 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:56.878749 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6073886a364274be529371945e0055', 'request_id': 'req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:56.886870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6073886a364274be529371945e0055', 'request_id': 'req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:57.066525 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6073886a364274be529371945e0055', 'request_id': 'req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:57.081343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] The token cache is being invalidate because project fb6073886a364274be529371945e0055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:57.085049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:57.085049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a035d4c9-3923-4d48-8b31-8f30cd1ccca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:57.085049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1311/2629] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:56 2022] DELETE /identity/v3/projects/fb6073886a364274be529371945e0055 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:57.089263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7d2da-9776-42af-8b94-4f79b9772d92 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.089965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7d2da-9776-42af-8b94-4f79b9772d92 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.090450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7d2da-9776-42af-8b94-4f79b9772d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:57.110137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8ba7d2da-9776-42af-8b94-4f79b9772d92 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:57.111847 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8ba7d2da-9776-42af-8b94-4f79b9772d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:57.119662 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:57.202015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1319/2630] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:57.206382 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb622f28-1e79-4b93-b4a1-26f8750fa2bd None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.206834 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb622f28-1e79-4b93-b4a1-26f8750fa2bd None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.206918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb622f28-1e79-4b93-b4a1-26f8750fa2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:57.229180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-eb622f28-1e79-4b93-b4a1-26f8750fa2bd None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:57.230341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-eb622f28-1e79-4b93-b4a1-26f8750fa2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:57.243417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1312/2631] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:03:57.244014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7d2da-9776-42af-8b94-4f79b9772d92 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:57.262155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7d2da-9776-42af-8b94-4f79b9772d92 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:57.268229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee9f37-7114-47f4-a0b0-925a02ae9f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:57.268744 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee9f37-7114-47f4-a0b0-925a02ae9f96 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.269036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee9f37-7114-47f4-a0b0-925a02ae9f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.269425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee9f37-7114-47f4-a0b0-925a02ae9f96 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:57.270229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee9f37-7114-47f4-a0b0-925a02ae9f96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:03:57.270540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee9f37-7114-47f4-a0b0-925a02ae9f96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:57.272156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee9f37-7114-47f4-a0b0-925a02ae9f96 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:57.295824 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1320/2632] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:03:57.296474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb622f28-1e79-4b93-b4a1-26f8750fa2bd None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:57.309099 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb622f28-1e79-4b93-b4a1-26f8750fa2bd None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:57.314069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfecb3-f20e-449c-8760-505e06b49720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:57.314926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfecb3-f20e-449c-8760-505e06b49720 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.315409 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfecb3-f20e-449c-8760-505e06b49720 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.315754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfecb3-f20e-449c-8760-505e06b49720 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:57.316605 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfecb3-f20e-449c-8760-505e06b49720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:57.319366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfecb3-f20e-449c-8760-505e06b49720 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:57.352408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:57.451793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1313/2633] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:03:57 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:57.455800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee9f37-7114-47f4-a0b0-925a02ae9f96 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:57.466790 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee9f37-7114-47f4-a0b0-925a02ae9f96 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:57.535526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002163-13d6-4cde-a88f-1fcfc849bcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:57.535791 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002163-13d6-4cde-a88f-1fcfc849bcde admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.536098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002163-13d6-4cde-a88f-1fcfc849bcde admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.536371 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002163-13d6-4cde-a88f-1fcfc849bcde admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:57.537074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002163-13d6-4cde-a88f-1fcfc849bcde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:57.539215 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002163-13d6-4cde-a88f-1fcfc849bcde admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:57.569758 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfecb3-f20e-449c-8760-505e06b49720 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:57.571841 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7ccfecb3-f20e-449c-8760-505e06b49720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:57.586583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfecb3-f20e-449c-8760-505e06b49720 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:57.594468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:57.595627 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.595986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.596335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] PATH_INFO: `/v3/users/1513d4077dd24a61965219535270f518` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:57.602133 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1513d4077dd24a61965219535270f518)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:57.606871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:57.698586 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1321/2634] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:03:57 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:03:57.703164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002163-13d6-4cde-a88f-1fcfc849bcde admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:57.718872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002163-13d6-4cde-a88f-1fcfc849bcde admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:57.733279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcee85-bc07-4cbe-bf0f-a7c60c897b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:57.734032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcee85-bc07-4cbe-bf0f-a7c60c897b7c admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.734332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcee85-bc07-4cbe-bf0f-a7c60c897b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.734617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcee85-bc07-4cbe-bf0f-a7c60c897b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:57.735362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcee85-bc07-4cbe-bf0f-a7c60c897b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:57.737035 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcee85-bc07-4cbe-bf0f-a7c60c897b7c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:57.764511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1322/2635] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:03:57.769853 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcee85-bc07-4cbe-bf0f-a7c60c897b7c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:57.784481 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcee85-bc07-4cbe-bf0f-a7c60c897b7c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:57.790453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81264d-6b11-4fe0-a17c-ac35047b830c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:57.791235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81264d-6b11-4fe0-a17c-ac35047b830c admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.791632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81264d-6b11-4fe0-a17c-ac35047b830c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.792174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81264d-6b11-4fe0-a17c-ac35047b830c admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:57.793628 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81264d-6b11-4fe0-a17c-ac35047b830c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:57.796049 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81264d-6b11-4fe0-a17c-ac35047b830c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:57.813293 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1323/2636] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:03:57.814522 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81264d-6b11-4fe0-a17c-ac35047b830c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:57.826868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81264d-6b11-4fe0-a17c-ac35047b830c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:57.838099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68463dc3-681a-4b25-ad8e-6819bbc42406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:57.840366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463dc3-681a-4b25-ad8e-6819bbc42406 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.840704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463dc3-681a-4b25-ad8e-6819bbc42406 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.841026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463dc3-681a-4b25-ad8e-6819bbc42406 admin admin] PATH_INFO: `/v3/projects/59b915690792451f8519082c4480009d/users/eaf17ff6d01c48b58de3c44d83fdab1a/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:57.875132 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68463dc3-681a-4b25-ad8e-6819bbc42406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b915690792451f8519082c4480009d, user_id=eaf17ff6d01c48b58de3c44d83fdab1a, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:57.879368 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68463dc3-681a-4b25-ad8e-6819bbc42406 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:57.938906 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1513d4077dd24a61965219535270f518'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:57.982276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1324/2637] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:03:57 2022] PUT /identity/v3/projects/59b915690792451f8519082c4480009d/users/eaf17ff6d01c48b58de3c44d83fdab1a/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:57.993003 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489d34a-ae92-45a6-a520-9cb250bcf724 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:57.993295 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489d34a-ae92-45a6-a520-9cb250bcf724 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:57.993579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489d34a-ae92-45a6-a520-9cb250bcf724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:58.038741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0489d34a-ae92-45a6-a520-9cb250bcf724 None None] MFA Rules not processed for user `eaf17ff6d01c48b58de3c44d83fdab1a`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:58.038741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0489d34a-ae92-45a6-a520-9cb250bcf724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:58.068366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1513d4077dd24a61965219535270f518'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:58.068366 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1513d4077dd24a61965219535270f518'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:58.080165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1513d4077dd24a61965219535270f518'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:58.081244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1513d4077dd24a61965219535270f518'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:58.113640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1314/2638] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:57 2022] DELETE /identity/v3/users/1513d4077dd24a61965219535270f518 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:03:58.113640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:58.116705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1325/2639] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:58.117949 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1883e0d-3478-4b83-885b-1f92b0874a96 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:58.118250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1883e0d-3478-4b83-885b-1f92b0874a96 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:58.119046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1883e0d-3478-4b83-885b-1f92b0874a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:58.119151 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:58.139018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-f1883e0d-3478-4b83-885b-1f92b0874a96 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:03:58.140376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f1883e0d-3478-4b83-885b-1f92b0874a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:58.200668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afa07c-8565-4ea8-aad8-b70ab1d013b7 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:58.206010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:58.206817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1326/2640] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:03:58.210016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:58.210016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:58.210016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] PATH_INFO: `/v3/users/3795aa6537d04bd58d6f0aa26e4376d4` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:58.210631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1883e0d-3478-4b83-885b-1f92b0874a96 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:58.210837 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3795aa6537d04bd58d6f0aa26e4376d4)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:58.211169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:58.217286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-f1883e0d-3478-4b83-885b-1f92b0874a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:58.265499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:58.280881 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1883e0d-3478-4b83-885b-1f92b0874a96 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:58.284664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ade11-5eab-452c-be83-8ecb609c2c07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5eeda3264191409eba35ffbb908770a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:58.285323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ade11-5eab-452c-be83-8ecb609c2c07 service glance] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:58.285605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ade11-5eab-452c-be83-8ecb609c2c07 service glance] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:58.285918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ade11-5eab-452c-be83-8ecb609c2c07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:58.350480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ade11-5eab-452c-be83-8ecb609c2c07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:58.351913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ade11-5eab-452c-be83-8ecb609c2c07 service glance] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:58.361311 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1327/2641] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:58.362319 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ade11-5eab-452c-be83-8ecb609c2c07 service glance] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:58.363680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-674ade11-5eab-452c-be83-8ecb609c2c07 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:58.426080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ade11-5eab-452c-be83-8ecb609c2c07 service glance] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:58.429928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654aac10-1679-49ac-b306-b1e2986ca74b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:58.430588 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654aac10-1679-49ac-b306-b1e2986ca74b service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:58.430871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654aac10-1679-49ac-b306-b1e2986ca74b service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:58.431241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654aac10-1679-49ac-b306-b1e2986ca74b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:58.442199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654aac10-1679-49ac-b306-b1e2986ca74b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:58.443809 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654aac10-1679-49ac-b306-b1e2986ca74b service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:58.455023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1328/2642] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:03:58.490302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3795aa6537d04bd58d6f0aa26e4376d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:58.638265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3795aa6537d04bd58d6f0aa26e4376d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:58.639024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3795aa6537d04bd58d6f0aa26e4376d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:58.648264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3795aa6537d04bd58d6f0aa26e4376d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:58.649276 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3795aa6537d04bd58d6f0aa26e4376d4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:58.673729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1315/2643] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:58 2022] DELETE /identity/v3/users/3795aa6537d04bd58d6f0aa26e4376d4 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:58.982568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654aac10-1679-49ac-b306-b1e2986ca74b service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:58.983611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:58.986670 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:58.994816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654aac10-1679-49ac-b306-b1e2986ca74b service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:59.069856 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:59.070845 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:59.071134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:59.071448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] PATH_INFO: `/v3/projects/b807646ffbc14c0e96acf2cf042ab9b8` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:59.072642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b807646ffbc14c0e96acf2cf042ab9b8)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:59.073947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354f37-0e9a-4806-a3cc-d5c74366bf33 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:59.075135 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:59.080190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:59.081114 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:59.081403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:59.081718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] PATH_INFO: `/v3/projects/634b7ca060fa457b9660f5a5680c27b6` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:59.082954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=634b7ca060fa457b9660f5a5680c27b6)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:59.085907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:59.259765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '634b7ca060fa457b9660f5a5680c27b6', 'request_id': 'req-e25d4605-f18b-4ed6-a92e-60574315c640'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.260928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b807646ffbc14c0e96acf2cf042ab9b8', 'request_id': 'req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.266113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '634b7ca060fa457b9660f5a5680c27b6', 'request_id': 'req-e25d4605-f18b-4ed6-a92e-60574315c640'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.266971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b807646ffbc14c0e96acf2cf042ab9b8', 'request_id': 'req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.357578 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '634b7ca060fa457b9660f5a5680c27b6', 'request_id': 'req-e25d4605-f18b-4ed6-a92e-60574315c640'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.378392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] The token cache is being invalidate because project 634b7ca060fa457b9660f5a5680c27b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:59.378392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.378392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.378392 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1316/2644] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:58 2022] DELETE /identity/v3/projects/634b7ca060fa457b9660f5a5680c27b6 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:59.381725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:59.383365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:59.443666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b807646ffbc14c0e96acf2cf042ab9b8', 'request_id': 'req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.451118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d4605-f18b-4ed6-a92e-60574315c640 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:59.456631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] The token cache is being invalidate because project b807646ffbc14c0e96acf2cf042ab9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:03:59.457037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.457772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.459705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1329/2645] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:03:58 2022] DELETE /identity/v3/projects/b807646ffbc14c0e96acf2cf042ab9b8 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:59.463560 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:03:59.465558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:03:59.511739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:59.512677 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:59.513071 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:59.513438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] PATH_INFO: `/v3/users/74fda6e0f4d84e06b96d20a35e0b5c0f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:59.514845 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=74fda6e0f4d84e06b96d20a35e0b5c0f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:59.517205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:59.533081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1ed98-dfd5-4f52-a84d-75cf5b2bff88 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:03:59.538365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:03:59.539376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:03:59.539587 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:03:59.539821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] PATH_INFO: `/v3/users/153d25021aaf4196a839068cf367ee31` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:03:59.543079 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=153d25021aaf4196a839068cf367ee31)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:03:59.545032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:03:59.575161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:59.798092 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74fda6e0f4d84e06b96d20a35e0b5c0f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.830046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153d25021aaf4196a839068cf367ee31'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.877655 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74fda6e0f4d84e06b96d20a35e0b5c0f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.878357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74fda6e0f4d84e06b96d20a35e0b5c0f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.893804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74fda6e0f4d84e06b96d20a35e0b5c0f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.893804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74fda6e0f4d84e06b96d20a35e0b5c0f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.919685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1317/2646] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:59 2022] DELETE /identity/v3/users/74fda6e0f4d84e06b96d20a35e0b5c0f => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:03:59.924134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:03:59.964027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153d25021aaf4196a839068cf367ee31'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.964969 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153d25021aaf4196a839068cf367ee31'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.974329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153d25021aaf4196a839068cf367ee31'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.975074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153d25021aaf4196a839068cf367ee31'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:03:59.995800 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1330/2647] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:03:59 2022] DELETE /identity/v3/users/153d25021aaf4196a839068cf367ee31 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:00.381639 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:00.383348 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:00.488181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e59dd0-32a0-4cce-90d2-4ebdd0b1a48a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:00.548077 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:00.549072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:00.549456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:00.549739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] PATH_INFO: `/v3/projects/f16c3e2474fd4e22b9ca883d93a80bc0` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:00.551005 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f16c3e2474fd4e22b9ca883d93a80bc0)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:00.552934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:00.632838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:00.634348 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:00.648750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb31d4-8a5f-40c9-bd01-650d4ad06668 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:00.654653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:00.655538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:00.655831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:00.656115 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] PATH_INFO: `/v3/projects/3ac344bccfad4e6a960a0c99bba2cada` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:00.657408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ac344bccfad4e6a960a0c99bba2cada)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:00.661290 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:00.723295 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f16c3e2474fd4e22b9ca883d93a80bc0', 'request_id': 'req-fe6e80da-5988-4bee-9bfc-db53412a666e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:00.728854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f16c3e2474fd4e22b9ca883d93a80bc0', 'request_id': 'req-fe6e80da-5988-4bee-9bfc-db53412a666e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:00.896170 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ac344bccfad4e6a960a0c99bba2cada', 'request_id': 'req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:00.901568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ac344bccfad4e6a960a0c99bba2cada', 'request_id': 'req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:00.912781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f16c3e2474fd4e22b9ca883d93a80bc0', 'request_id': 'req-fe6e80da-5988-4bee-9bfc-db53412a666e'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:00.926028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] The token cache is being invalidate because project f16c3e2474fd4e22b9ca883d93a80bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:00.926375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:00.927319 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-fe6e80da-5988-4bee-9bfc-db53412a666e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:00.929031 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1318/2648] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:00 2022] DELETE /identity/v3/projects/f16c3e2474fd4e22b9ca883d93a80bc0 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:00.934763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013e71a-88bd-4ed5-9402-1bde3a45a01e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:00.935053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013e71a-88bd-4ed5-9402-1bde3a45a01e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:00.935333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013e71a-88bd-4ed5-9402-1bde3a45a01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:00.955503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-0013e71a-88bd-4ed5-9402-1bde3a45a01e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:00.956750 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-0013e71a-88bd-4ed5-9402-1bde3a45a01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:01.021486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1319/2649] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:01.029202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffcd54-3c45-4300-8ad6-81afe9fefad3 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.029490 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffcd54-3c45-4300-8ad6-81afe9fefad3 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.029764 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffcd54-3c45-4300-8ad6-81afe9fefad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.049453 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e5ffcd54-3c45-4300-8ad6-81afe9fefad3 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:01.050574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e5ffcd54-3c45-4300-8ad6-81afe9fefad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:01.060852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1320/2650] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:01.065324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffcd54-3c45-4300-8ad6-81afe9fefad3 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:01.071740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ac344bccfad4e6a960a0c99bba2cada', 'request_id': 'req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:01.076309 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffcd54-3c45-4300-8ad6-81afe9fefad3 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:01.083279 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:01.083941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.084226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.084504 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.085241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:01.085618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:01.086905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] The token cache is being invalidate because project 3ac344bccfad4e6a960a0c99bba2cada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:01.089728 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:01.090348 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:01.090447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:01.092681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1331/2651] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:00 2022] DELETE /identity/v3/projects/3ac344bccfad4e6a960a0c99bba2cada => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:01.093581 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:01.094983 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:01.103920 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1321/2652] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:04:01.104924 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:01.107185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:01.108365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3cbc-1d42-42fb-b1f8-c096e11b2fa6 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:01.113431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:01.114375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.114668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.114946 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] PATH_INFO: `/v3/users/5b72aa635501455bac35a97646e047c8` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.116195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b72aa635501455bac35a97646e047c8)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:01.118268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:01.121722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e647f00-20b0-4143-8c37-59070f85a028 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:01.127451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad9c9-6e89-496f-ac6c-c6dccb552d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:01.135445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad9c9-6e89-496f-ac6c-c6dccb552d83 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.135445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad9c9-6e89-496f-ac6c-c6dccb552d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.135445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ad9c9-6e89-496f-ac6c-c6dccb552d83 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.135445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ad9c9-6e89-496f-ac6c-c6dccb552d83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:01.135445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ad9c9-6e89-496f-ac6c-c6dccb552d83 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:01.221356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:01.230029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1322/2653] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:01 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:01.231662 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad9c9-6e89-496f-ac6c-c6dccb552d83 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:01.294957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ad9c9-6e89-496f-ac6c-c6dccb552d83 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:01.300623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b981167-8603-4fb6-8658-3e73b51ff54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:01.301281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b981167-8603-4fb6-8658-3e73b51ff54f admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.301566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b981167-8603-4fb6-8658-3e73b51ff54f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.301865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b981167-8603-4fb6-8658-3e73b51ff54f admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.302562 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b981167-8603-4fb6-8658-3e73b51ff54f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:01.304589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b981167-8603-4fb6-8658-3e73b51ff54f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:01.366976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b72aa635501455bac35a97646e047c8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:01.396821 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1323/2654] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:01 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:01.401134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b981167-8603-4fb6-8658-3e73b51ff54f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:01.411690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b981167-8603-4fb6-8658-3e73b51ff54f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:01.416337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6233cfe-0b0c-40cd-81ee-2a9f18255e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:01.417028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6233cfe-0b0c-40cd-81ee-2a9f18255e08 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.417307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6233cfe-0b0c-40cd-81ee-2a9f18255e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.418540 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6233cfe-0b0c-40cd-81ee-2a9f18255e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.418946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6233cfe-0b0c-40cd-81ee-2a9f18255e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:01.420671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6233cfe-0b0c-40cd-81ee-2a9f18255e08 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:01.431565 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1324/2655] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:01.435785 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6233cfe-0b0c-40cd-81ee-2a9f18255e08 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:01.445538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6233cfe-0b0c-40cd-81ee-2a9f18255e08 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:01.450363 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d52e8-bbb7-4a69-8b57-942ce0ebb75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:01.451068 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d52e8-bbb7-4a69-8b57-942ce0ebb75a admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.451340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d52e8-bbb7-4a69-8b57-942ce0ebb75a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.451623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d52e8-bbb7-4a69-8b57-942ce0ebb75a admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.452433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d52e8-bbb7-4a69-8b57-942ce0ebb75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:01.454094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d52e8-bbb7-4a69-8b57-942ce0ebb75a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:01.465838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1325/2656] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:01.469936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d52e8-bbb7-4a69-8b57-942ce0ebb75a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:01.481433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d52e8-bbb7-4a69-8b57-942ce0ebb75a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:01.483216 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b72aa635501455bac35a97646e047c8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:01.483812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b72aa635501455bac35a97646e047c8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:01.486668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9118-35a6-4c8c-ad44-8ff33c0297e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:01.487603 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9118-35a6-4c8c-ad44-8ff33c0297e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.487888 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9118-35a6-4c8c-ad44-8ff33c0297e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.488287 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9118-35a6-4c8c-ad44-8ff33c0297e7 admin admin] PATH_INFO: `/v3/projects/2f8ba7acb9ba4ca1b2d27b73aeb6996c/users/24b52a9dbf5148f6afdce42e7f305b4c/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.492479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b72aa635501455bac35a97646e047c8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:01.493336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-b3a64501-384e-415f-abaa-a583c408f3ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b72aa635501455bac35a97646e047c8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:01.507718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf9118-35a6-4c8c-ad44-8ff33c0297e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8ba7acb9ba4ca1b2d27b73aeb6996c, user_id=24b52a9dbf5148f6afdce42e7f305b4c, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:01.510999 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf9118-35a6-4c8c-ad44-8ff33c0297e7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:01.516966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1332/2657] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:01 2022] DELETE /identity/v3/users/5b72aa635501455bac35a97646e047c8 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:01.616174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1326/2658] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:01 2022] PUT /identity/v3/projects/2f8ba7acb9ba4ca1b2d27b73aeb6996c/users/24b52a9dbf5148f6afdce42e7f305b4c/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:01.626784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5e4dc-6d54-4243-a113-c889f5aec79a None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.627158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5e4dc-6d54-4243-a113-c889f5aec79a None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.627642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5e4dc-6d54-4243-a113-c889f5aec79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.657252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-36e5e4dc-6d54-4243-a113-c889f5aec79a None None] MFA Rules not processed for user `24b52a9dbf5148f6afdce42e7f305b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:01.658609 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-36e5e4dc-6d54-4243-a113-c889f5aec79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:01.740356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1333/2659] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:01.745000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9118-35a6-4c8c-ad44-8ff33c0297e7 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:01.746916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-23bf9118-35a6-4c8c-ad44-8ff33c0297e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:01.816325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9118-35a6-4c8c-ad44-8ff33c0297e7 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:01.821344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f0b26-7d60-4ae6-8660-39c4df23916b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:01.822052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f0b26-7d60-4ae6-8660-39c4df23916b admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:01.822344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f0b26-7d60-4ae6-8660-39c4df23916b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:01.822613 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f0b26-7d60-4ae6-8660-39c4df23916b admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:01.823333 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f0b26-7d60-4ae6-8660-39c4df23916b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:01.825871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f0b26-7d60-4ae6-8660-39c4df23916b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.048619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1327/2660] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:01 2022] POST /identity/v3/projects => generated 387 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:02.052919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5e4dc-6d54-4243-a113-c889f5aec79a None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.066195 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5e4dc-6d54-4243-a113-c889f5aec79a None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.130704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c3c5d-a9a8-4c78-8fb4-6d15f9545c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.131594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c3c5d-a9a8-4c78-8fb4-6d15f9545c41 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.131900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c3c5d-a9a8-4c78-8fb4-6d15f9545c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.132180 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c3c5d-a9a8-4c78-8fb4-6d15f9545c41 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.132876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c3c5d-a9a8-4c78-8fb4-6d15f9545c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.134827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c3c5d-a9a8-4c78-8fb4-6d15f9545c41 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.284797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1334/2661] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:02 2022] POST /identity/v3/users => generated 300 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:02.288931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f0b26-7d60-4ae6-8660-39c4df23916b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.300553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f0b26-7d60-4ae6-8660-39c4df23916b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.305589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd305aca-209a-4105-8c8c-40f4bb1b067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.306293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd305aca-209a-4105-8c8c-40f4bb1b067d admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.306582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd305aca-209a-4105-8c8c-40f4bb1b067d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.306857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd305aca-209a-4105-8c8c-40f4bb1b067d admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.307580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd305aca-209a-4105-8c8c-40f4bb1b067d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.309324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd305aca-209a-4105-8c8c-40f4bb1b067d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.322429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1328/2662] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:02.327023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c3c5d-a9a8-4c78-8fb4-6d15f9545c41 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.337280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c3c5d-a9a8-4c78-8fb4-6d15f9545c41 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.343096 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577a74c-953a-4b67-ab20-cbb34116e273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.344045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577a74c-953a-4b67-ab20-cbb34116e273 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.344338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577a74c-953a-4b67-ab20-cbb34116e273 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.344637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577a74c-953a-4b67-ab20-cbb34116e273 admin admin] PATH_INFO: `/v3/projects/9a6675fd71014a44aa2d5fc1332109d6/users/6fcf5e6991c64e229a11fc821d0b4093/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.354100 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd305aca-209a-4105-8c8c-40f4bb1b067d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.357730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-dd305aca-209a-4105-8c8c-40f4bb1b067d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:02.362870 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577a74c-953a-4b67-ab20-cbb34116e273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6675fd71014a44aa2d5fc1332109d6, user_id=6fcf5e6991c64e229a11fc821d0b4093, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.365612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577a74c-953a-4b67-ab20-cbb34116e273 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.375573 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd305aca-209a-4105-8c8c-40f4bb1b067d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.382534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.383460 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.383936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.384224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] PATH_INFO: `/v3/projects/4592f3be4eab4323979f321efa1aaff1` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.385861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4592f3be4eab4323979f321efa1aaff1)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.391521 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.471804 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1335/2663] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:02 2022] PUT /identity/v3/projects/9a6675fd71014a44aa2d5fc1332109d6/users/6fcf5e6991c64e229a11fc821d0b4093/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:02.476060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577a74c-953a-4b67-ab20-cbb34116e273 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.545166 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577a74c-953a-4b67-ab20-cbb34116e273 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.550167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70208f-50d3-4a5e-ae7f-9f4aa121e5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.550855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70208f-50d3-4a5e-ae7f-9f4aa121e5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.551145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70208f-50d3-4a5e-ae7f-9f4aa121e5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.551438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70208f-50d3-4a5e-ae7f-9f4aa121e5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.552647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70208f-50d3-4a5e-ae7f-9f4aa121e5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.553795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70208f-50d3-4a5e-ae7f-9f4aa121e5d9 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.564858 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1336/2664] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:02.571439 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70208f-50d3-4a5e-ae7f-9f4aa121e5d9 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.586705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70208f-50d3-4a5e-ae7f-9f4aa121e5d9 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.589647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4592f3be4eab4323979f321efa1aaff1', 'request_id': 'req-09c20b1d-badb-4c8d-b118-cf360da7c107'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:02.596537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4592f3be4eab4323979f321efa1aaff1', 'request_id': 'req-09c20b1d-badb-4c8d-b118-cf360da7c107'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:02.650828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb74468-4f4a-481a-8fd2-18d565bbfe87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.651735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb74468-4f4a-481a-8fd2-18d565bbfe87 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.652014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb74468-4f4a-481a-8fd2-18d565bbfe87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.652863 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb74468-4f4a-481a-8fd2-18d565bbfe87 admin admin] PATH_INFO: `/v3/projects/9a6675fd71014a44aa2d5fc1332109d6/users/6fcf5e6991c64e229a11fc821d0b4093/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.655017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb74468-4f4a-481a-8fd2-18d565bbfe87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6675fd71014a44aa2d5fc1332109d6, user_id=6fcf5e6991c64e229a11fc821d0b4093, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.658010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb74468-4f4a-481a-8fd2-18d565bbfe87 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.668700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1337/2665] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:02 2022] PUT /identity/v3/projects/9a6675fd71014a44aa2d5fc1332109d6/users/6fcf5e6991c64e229a11fc821d0b4093/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:02.680398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6035c-9eeb-4c0e-918c-8b18d94e25bf None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.680690 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6035c-9eeb-4c0e-918c-8b18d94e25bf None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.681011 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6035c-9eeb-4c0e-918c-8b18d94e25bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.710807 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-0fe6035c-9eeb-4c0e-918c-8b18d94e25bf None None] MFA Rules not processed for user `6fcf5e6991c64e229a11fc821d0b4093`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:02.711489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4592f3be4eab4323979f321efa1aaff1', 'request_id': 'req-09c20b1d-badb-4c8d-b118-cf360da7c107'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:02.712026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0fe6035c-9eeb-4c0e-918c-8b18d94e25bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:02.725200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] The token cache is being invalidate because project 4592f3be4eab4323979f321efa1aaff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:02.725793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:02.726634 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-09c20b1d-badb-4c8d-b118-cf360da7c107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:02.728450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1329/2666] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:02 2022] DELETE /identity/v3/projects/4592f3be4eab4323979f321efa1aaff1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:02.736049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a26b5-4453-4e93-9d4c-3beab0ea417e None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.736365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a26b5-4453-4e93-9d4c-3beab0ea417e None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.736640 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a26b5-4453-4e93-9d4c-3beab0ea417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.755812 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-052a26b5-4453-4e93-9d4c-3beab0ea417e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:02.758891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-052a26b5-4453-4e93-9d4c-3beab0ea417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:02.787056 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:02.798854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1338/2667] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:02.810622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6035c-9eeb-4c0e-918c-8b18d94e25bf None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.813871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-0fe6035c-9eeb-4c0e-918c-8b18d94e25bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:02.825843 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1330/2668] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:02.835147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e1f8-ab65-4bd3-a20f-5dd772cc5dfb None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.835899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e1f8-ab65-4bd3-a20f-5dd772cc5dfb None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.836106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e1f8-ab65-4bd3-a20f-5dd772cc5dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.855618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f1d2e1f8-ab65-4bd3-a20f-5dd772cc5dfb None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:02.858458 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f1d2e1f8-ab65-4bd3-a20f-5dd772cc5dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:02.868297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1331/2669] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:02.872126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6035c-9eeb-4c0e-918c-8b18d94e25bf None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.872462 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2e1f8-ab65-4bd3-a20f-5dd772cc5dfb None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.875907 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191eb1-910d-4bb3-a75b-9dccfb2bd22f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.876592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191eb1-910d-4bb3-a75b-9dccfb2bd22f service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.876875 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191eb1-910d-4bb3-a75b-9dccfb2bd22f service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.877155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191eb1-910d-4bb3-a75b-9dccfb2bd22f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.882091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2e1f8-ab65-4bd3-a20f-5dd772cc5dfb None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.887700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.888383 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.888660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.888931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.889234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48191eb1-910d-4bb3-a75b-9dccfb2bd22f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.889661 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:02.889995 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.891024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48191eb1-910d-4bb3-a75b-9dccfb2bd22f service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.891624 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.901201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1339/2670] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:04:02.906653 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1332/2671] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:04:02.911342 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191eb1-910d-4bb3-a75b-9dccfb2bd22f service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.923839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191eb1-910d-4bb3-a75b-9dccfb2bd22f service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.928850 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f00a3-1ae4-49ed-bf76-c649fc4f7aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.930734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f00a3-1ae4-49ed-bf76-c649fc4f7aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.930734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f00a3-1ae4-49ed-bf76-c649fc4f7aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.930734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f00a3-1ae4-49ed-bf76-c649fc4f7aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.931086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f00a3-1ae4-49ed-bf76-c649fc4f7aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.932654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:02.934726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f00a3-1ae4-49ed-bf76-c649fc4f7aa1 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:02.935176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:02.952262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28ef4a-71f1-4f58-a4ab-d8996f223b55 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:02.957320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:02.960318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:02.960318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:02.960318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] PATH_INFO: `/v3/users/24b52a9dbf5148f6afdce42e7f305b4c` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:02.960318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b52a9dbf5148f6afdce42e7f305b4c)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:02.962035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:03.022454 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1340/2672] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:02 2022] POST /identity/v3/projects => generated 361 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:03.026829 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f00a3-1ae4-49ed-bf76-c649fc4f7aa1 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:03.091793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f00a3-1ae4-49ed-bf76-c649fc4f7aa1 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:03.097630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af4e30-4691-4fc8-bddf-9b37fa8e87cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:03.098366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af4e30-4691-4fc8-bddf-9b37fa8e87cc admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:03.098677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af4e30-4691-4fc8-bddf-9b37fa8e87cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:03.103726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af4e30-4691-4fc8-bddf-9b37fa8e87cc admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:03.103726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af4e30-4691-4fc8-bddf-9b37fa8e87cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:03.103726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af4e30-4691-4fc8-bddf-9b37fa8e87cc admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:03.225214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1341/2673] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:03 2022] POST /identity/v3/users => generated 281 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:03.229377 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af4e30-4691-4fc8-bddf-9b37fa8e87cc admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:03.239521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af4e30-4691-4fc8-bddf-9b37fa8e87cc admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:03.244466 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba494c-c725-4b9e-844c-8f28fc2459e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:03.245223 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba494c-c725-4b9e-844c-8f28fc2459e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:03.245511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba494c-c725-4b9e-844c-8f28fc2459e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:03.245789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba494c-c725-4b9e-844c-8f28fc2459e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:03.247088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba494c-c725-4b9e-844c-8f28fc2459e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:03.248699 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba494c-c725-4b9e-844c-8f28fc2459e0 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:03.256273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=72,keystone:DELETE=25,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:03.264835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1342/2674] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:03.268274 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba494c-c725-4b9e-844c-8f28fc2459e0 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:03.271415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b52a9dbf5148f6afdce42e7f305b4c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:03.276226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba494c-c725-4b9e-844c-8f28fc2459e0 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:03.281497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafca32-9289-44dc-adb6-8704d76be9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:03.282208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafca32-9289-44dc-adb6-8704d76be9cc admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:03.282497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafca32-9289-44dc-adb6-8704d76be9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:03.282776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafca32-9289-44dc-adb6-8704d76be9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:03.283636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafca32-9289-44dc-adb6-8704d76be9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:03.287585 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafca32-9289-44dc-adb6-8704d76be9cc admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:03.299606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1343/2675] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:03.317419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafca32-9289-44dc-adb6-8704d76be9cc admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:03.329552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafca32-9289-44dc-adb6-8704d76be9cc admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:03.334798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6306f-6156-4af7-87bb-0a69a7770407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:03.335724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6306f-6156-4af7-87bb-0a69a7770407 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:03.336008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6306f-6156-4af7-87bb-0a69a7770407 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:03.336282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6306f-6156-4af7-87bb-0a69a7770407 admin admin] PATH_INFO: `/v3/projects/bcbceab4c5e241029635f4910c055cb9/users/5c006e56b0e84e999d7aecca29cb8727/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:03.354042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6306f-6156-4af7-87bb-0a69a7770407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbceab4c5e241029635f4910c055cb9, user_id=5c006e56b0e84e999d7aecca29cb8727, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:03.356392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6306f-6156-4af7-87bb-0a69a7770407 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:03.374903 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b52a9dbf5148f6afdce42e7f305b4c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:03.375611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b52a9dbf5148f6afdce42e7f305b4c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:03.385972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b52a9dbf5148f6afdce42e7f305b4c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:03.386779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-c8d07a53-ff3e-44ee-974d-18267b280d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b52a9dbf5148f6afdce42e7f305b4c'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:03.404790 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1333/2676] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:02 2022] DELETE /identity/v3/users/24b52a9dbf5148f6afdce42e7f305b4c => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:03.409672 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a19da-001c-4b85-8bc8-f87ab206f79b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:03.410022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a19da-001c-4b85-8bc8-f87ab206f79b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:03.410305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a19da-001c-4b85-8bc8-f87ab206f79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:03.428569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-1e7a19da-001c-4b85-8bc8-f87ab206f79b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:03.429803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1e7a19da-001c-4b85-8bc8-f87ab206f79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:03.490492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1334/2677] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:03.501169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a19da-001c-4b85-8bc8-f87ab206f79b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:03.502596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1e7a19da-001c-4b85-8bc8-f87ab206f79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:03.517137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1344/2678] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:03 2022] PUT /identity/v3/projects/bcbceab4c5e241029635f4910c055cb9/users/5c006e56b0e84e999d7aecca29cb8727/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:03.530428 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c96b2-2f80-4d92-bb92-ed8ad7aad0ac None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:03.530731 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c96b2-2f80-4d92-bb92-ed8ad7aad0ac None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:03.531000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c96b2-2f80-4d92-bb92-ed8ad7aad0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:03.559210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-051c96b2-2f80-4d92-bb92-ed8ad7aad0ac None None] MFA Rules not processed for user `5c006e56b0e84e999d7aecca29cb8727`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:03.566085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-051c96b2-2f80-4d92-bb92-ed8ad7aad0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:03.566224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a19da-001c-4b85-8bc8-f87ab206f79b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:03.623241 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52de8-6333-4d53-9112-df0063c4a6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:03.624007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52de8-6333-4d53-9112-df0063c4a6aa service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:03.624354 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52de8-6333-4d53-9112-df0063c4a6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:03.624700 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52de8-6333-4d53-9112-df0063c4a6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:03.645379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1345/2679] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:03.684078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c96b2-2f80-4d92-bb92-ed8ad7aad0ac None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:03.686489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-051c96b2-2f80-4d92-bb92-ed8ad7aad0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:03.714622 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52de8-6333-4d53-9112-df0063c4a6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:03.716335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52de8-6333-4d53-9112-df0063c4a6aa service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:03.728588 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1335/2680] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:04:03.740355 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:03.758399 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c96b2-2f80-4d92-bb92-ed8ad7aad0ac None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:03.762976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a76eea-8727-4f6a-b489-6c5af0c8bbb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:03.763646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76eea-8727-4f6a-b489-6c5af0c8bbb1 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:03.763933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76eea-8727-4f6a-b489-6c5af0c8bbb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:03.764212 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a76eea-8727-4f6a-b489-6c5af0c8bbb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:03.773126 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a76eea-8727-4f6a-b489-6c5af0c8bbb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:03.775500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a76eea-8727-4f6a-b489-6c5af0c8bbb1 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:03.789887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1346/2681] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:04:03.804297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52de8-6333-4d53-9112-df0063c4a6aa service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:03.805855 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a5e52de8-6333-4d53-9112-df0063c4a6aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:03.820165 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52de8-6333-4d53-9112-df0063c4a6aa service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:03.830110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:03.830110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:03.830110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:03.830110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] PATH_INFO: `/v3/users/5c006e56b0e84e999d7aecca29cb8727` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:03.830767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c006e56b0e84e999d7aecca29cb8727)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:03.830767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:03.986606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:04.029781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c006e56b0e84e999d7aecca29cb8727'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.110861 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c006e56b0e84e999d7aecca29cb8727'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.112152 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c006e56b0e84e999d7aecca29cb8727'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.124394 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c006e56b0e84e999d7aecca29cb8727'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.125236 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c006e56b0e84e999d7aecca29cb8727'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.147070 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1336/2682] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:03 2022] DELETE /identity/v3/users/5c006e56b0e84e999d7aecca29cb8727 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:04.151950 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7fcfb-a52f-49c4-bc9b-708bdd9fbd54 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:04.152353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7fcfb-a52f-49c4-bc9b-708bdd9fbd54 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:04.152635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7fcfb-a52f-49c4-bc9b-708bdd9fbd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:04.172210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-66c7fcfb-a52f-49c4-bc9b-708bdd9fbd54 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:04.173381 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-66c7fcfb-a52f-49c4-bc9b-708bdd9fbd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:04.237514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1347/2683] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:04.248887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:04.250914 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:04.268887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7fcfb-a52f-49c4-bc9b-708bdd9fbd54 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:04.270209 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-66c7fcfb-a52f-49c4-bc9b-708bdd9fbd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:04.273788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:04.285280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7fcfb-a52f-49c4-bc9b-708bdd9fbd54 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:04.290784 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:04.291694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:04.291982 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:04.292263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] PATH_INFO: `/v3/projects/2f8ba7acb9ba4ca1b2d27b73aeb6996c` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:04.299772 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8ba7acb9ba4ca1b2d27b73aeb6996c)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:04.301725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:04.322713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba4f60-1d2b-4552-817c-b2cf4b9fe023 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:04.326690 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f350b8-0bf4-4ad4-9ecb-69ce7de382e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:04.327399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f350b8-0bf4-4ad4-9ecb-69ce7de382e6 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:04.327689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f350b8-0bf4-4ad4-9ecb-69ce7de382e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:04.327977 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f350b8-0bf4-4ad4-9ecb-69ce7de382e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:04.337865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f350b8-0bf4-4ad4-9ecb-69ce7de382e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:04.339297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f350b8-0bf4-4ad4-9ecb-69ce7de382e6 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:04.349263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1337/2684] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:04.453321 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8ba7acb9ba4ca1b2d27b73aeb6996c', 'request_id': 'req-e5d5a691-70d3-4eba-8466-a8e351c92e32'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.459366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8ba7acb9ba4ca1b2d27b73aeb6996c', 'request_id': 'req-e5d5a691-70d3-4eba-8466-a8e351c92e32'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.529352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8ba7acb9ba4ca1b2d27b73aeb6996c', 'request_id': 'req-e5d5a691-70d3-4eba-8466-a8e351c92e32'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.542733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] The token cache is being invalidate because project 2f8ba7acb9ba4ca1b2d27b73aeb6996c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:04.543837 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.544654 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.546338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1348/2685] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:04 2022] DELETE /identity/v3/projects/2f8ba7acb9ba4ca1b2d27b73aeb6996c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:04.551207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f350b8-0bf4-4ad4-9ecb-69ce7de382e6 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:04.561315 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f350b8-0bf4-4ad4-9ecb-69ce7de382e6 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:04.609734 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:04.631328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:04.632337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:04.632630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:04.632911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] PATH_INFO: `/v3/users/6fcf5e6991c64e229a11fc821d0b4093` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:04.634200 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fcf5e6991c64e229a11fc821d0b4093)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:04.638269 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:04.845064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fcf5e6991c64e229a11fc821d0b4093'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.964434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fcf5e6991c64e229a11fc821d0b4093'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.965527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fcf5e6991c64e229a11fc821d0b4093'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.975800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fcf5e6991c64e229a11fc821d0b4093'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.975800 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-93bf06f2-0501-4d8c-9c95-7e1734da2263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fcf5e6991c64e229a11fc821d0b4093'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:04.994880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1338/2686] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:04 2022] DELETE /identity/v3/users/6fcf5e6991c64e229a11fc821d0b4093 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:05.101618 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:05.105743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:05.122047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5a691-70d3-4eba-8466-a8e351c92e32 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:05.231771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:05.232955 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:05.233285 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:05.233578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] PATH_INFO: `/v3/projects/bcbceab4c5e241029635f4910c055cb9` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:05.235894 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcbceab4c5e241029635f4910c055cb9)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:05.237877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:05.318667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:05.449970 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcbceab4c5e241029635f4910c055cb9', 'request_id': 'req-361be5e6-ce1b-4995-9c85-79ae28ddd35b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:05.456502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcbceab4c5e241029635f4910c055cb9', 'request_id': 'req-361be5e6-ce1b-4995-9c85-79ae28ddd35b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:05.598742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcbceab4c5e241029635f4910c055cb9', 'request_id': 'req-361be5e6-ce1b-4995-9c85-79ae28ddd35b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:05.613544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] The token cache is being invalidate because project bcbceab4c5e241029635f4910c055cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:05.613872 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:05.614550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:05.616508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1349/2687] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:05 2022] DELETE /identity/v3/projects/bcbceab4c5e241029635f4910c055cb9 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:05.621897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53b02f-070a-4ce4-9486-1f3b52b250bf None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:05.622216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53b02f-070a-4ce4-9486-1f3b52b250bf None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:05.622524 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53b02f-070a-4ce4-9486-1f3b52b250bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:05.643465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-be53b02f-070a-4ce4-9486-1f3b52b250bf None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:05.644741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-be53b02f-070a-4ce4-9486-1f3b52b250bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:05.655164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:05.662398 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:05.663851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:05.719302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1339/2688] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:05.727229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb19a81-2a68-4b49-976d-f0d39938875c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:05.727891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb19a81-2a68-4b49-976d-f0d39938875c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:05.728174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb19a81-2a68-4b49-976d-f0d39938875c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:05.738433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361be5e6-ce1b-4995-9c85-79ae28ddd35b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:05.743258 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:05.744071 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:05.744352 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:05.744633 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] PATH_INFO: `/v3/projects/9a6675fd71014a44aa2d5fc1332109d6` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:05.745971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a6675fd71014a44aa2d5fc1332109d6)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:05.747418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-adb19a81-2a68-4b49-976d-f0d39938875c None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:05.749912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:05.750079 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-adb19a81-2a68-4b49-976d-f0d39938875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:05.761950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1340/2689] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:05.762600 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb19a81-2a68-4b49-976d-f0d39938875c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:05.772712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb19a81-2a68-4b49-976d-f0d39938875c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:05.784060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165e252-daf5-495d-bdef-8be2147d8289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:05.784666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165e252-daf5-495d-bdef-8be2147d8289 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:05.784817 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165e252-daf5-495d-bdef-8be2147d8289 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:05.785091 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165e252-daf5-495d-bdef-8be2147d8289 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:05.785842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165e252-daf5-495d-bdef-8be2147d8289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:05.786305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165e252-daf5-495d-bdef-8be2147d8289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:05.788123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165e252-daf5-495d-bdef-8be2147d8289 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:05.806044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1341/2690] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:04:05.807119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165e252-daf5-495d-bdef-8be2147d8289 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:05.818641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165e252-daf5-495d-bdef-8be2147d8289 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:05.825197 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345e015-930b-45c1-b69f-6b8403f95653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:05.825919 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345e015-930b-45c1-b69f-6b8403f95653 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:05.826210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345e015-930b-45c1-b69f-6b8403f95653 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:05.826492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345e015-930b-45c1-b69f-6b8403f95653 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:05.827262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345e015-930b-45c1-b69f-6b8403f95653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:05.829199 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345e015-930b-45c1-b69f-6b8403f95653 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:05.925942 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a6675fd71014a44aa2d5fc1332109d6', 'request_id': 'req-63150ba2-aea7-4b17-8840-86ee00ff843e'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:05.931697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a6675fd71014a44aa2d5fc1332109d6', 'request_id': 'req-63150ba2-aea7-4b17-8840-86ee00ff843e'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:06.083066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a6675fd71014a44aa2d5fc1332109d6', 'request_id': 'req-63150ba2-aea7-4b17-8840-86ee00ff843e'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:06.092044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1342/2691] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:05 2022] POST /identity/v3/projects => generated 385 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:06.096316 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345e015-930b-45c1-b69f-6b8403f95653 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.097131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] The token cache is being invalidate because project 9a6675fd71014a44aa2d5fc1332109d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:06.097552 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:06.098286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-63150ba2-aea7-4b17-8840-86ee00ff843e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:06.100093 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1350/2692] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:05 2022] DELETE /identity/v3/projects/9a6675fd71014a44aa2d5fc1332109d6 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:06.104801 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f091dfb-1517-436d-9600-4e4094aad0e9 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.105409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f091dfb-1517-436d-9600-4e4094aad0e9 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.105409 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f091dfb-1517-436d-9600-4e4094aad0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.130074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-1f091dfb-1517-436d-9600-4e4094aad0e9 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:06.131539 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1f091dfb-1517-436d-9600-4e4094aad0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:06.162669 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345e015-930b-45c1-b69f-6b8403f95653 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.168646 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890176c4-d4a2-4db1-b30b-1ff15f2f56b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.169324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890176c4-d4a2-4db1-b30b-1ff15f2f56b1 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.169610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890176c4-d4a2-4db1-b30b-1ff15f2f56b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.169916 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890176c4-d4a2-4db1-b30b-1ff15f2f56b1 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.170632 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890176c4-d4a2-4db1-b30b-1ff15f2f56b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.172985 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890176c4-d4a2-4db1-b30b-1ff15f2f56b1 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.197026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1351/2693] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:06.205347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc665828-eb9b-49a5-92a1-db57e9b193cb None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.213971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc665828-eb9b-49a5-92a1-db57e9b193cb None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.213971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc665828-eb9b-49a5-92a1-db57e9b193cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.228785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-fc665828-eb9b-49a5-92a1-db57e9b193cb None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:06.230413 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fc665828-eb9b-49a5-92a1-db57e9b193cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:06.238828 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1352/2694] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:06.243092 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc665828-eb9b-49a5-92a1-db57e9b193cb None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.252514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc665828-eb9b-49a5-92a1-db57e9b193cb None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.257239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fc8f8-9066-4e95-b2ee-e128fcf9d080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.259678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fc8f8-9066-4e95-b2ee-e128fcf9d080 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.259678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fc8f8-9066-4e95-b2ee-e128fcf9d080 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.259678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fc8f8-9066-4e95-b2ee-e128fcf9d080 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.259678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fc8f8-9066-4e95-b2ee-e128fcf9d080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:06.259678 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fc8f8-9066-4e95-b2ee-e128fcf9d080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.261122 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fc8f8-9066-4e95-b2ee-e128fcf9d080 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.275488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1353/2695] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:04:06.279627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fc8f8-9066-4e95-b2ee-e128fcf9d080 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.290568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fc8f8-9066-4e95-b2ee-e128fcf9d080 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.295964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76066c14-5e54-46dd-8305-b2c6de04e65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.296884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76066c14-5e54-46dd-8305-b2c6de04e65c admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.297168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76066c14-5e54-46dd-8305-b2c6de04e65c admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.298782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76066c14-5e54-46dd-8305-b2c6de04e65c admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.298782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76066c14-5e54-46dd-8305-b2c6de04e65c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.303490 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76066c14-5e54-46dd-8305-b2c6de04e65c admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.385586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1343/2696] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:06 2022] POST /identity/v3/users => generated 293 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:06.400383 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890176c4-d4a2-4db1-b30b-1ff15f2f56b1 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.402259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-890176c4-d4a2-4db1-b30b-1ff15f2f56b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:06.414567 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890176c4-d4a2-4db1-b30b-1ff15f2f56b1 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.419403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57bbca-487d-4fbf-b532-acd44b4b2064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.420591 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57bbca-487d-4fbf-b532-acd44b4b2064 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.420872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57bbca-487d-4fbf-b532-acd44b4b2064 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.421144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57bbca-487d-4fbf-b532-acd44b4b2064 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.422257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57bbca-487d-4fbf-b532-acd44b4b2064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.423971 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57bbca-487d-4fbf-b532-acd44b4b2064 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.434760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1344/2697] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:06.439344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57bbca-487d-4fbf-b532-acd44b4b2064 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.449291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57bbca-487d-4fbf-b532-acd44b4b2064 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.453990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee183e7-510c-442a-bf00-3eb446f9c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.454717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee183e7-510c-442a-bf00-3eb446f9c968 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.455004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee183e7-510c-442a-bf00-3eb446f9c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.455331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee183e7-510c-442a-bf00-3eb446f9c968 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.456101 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee183e7-510c-442a-bf00-3eb446f9c968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.457852 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee183e7-510c-442a-bf00-3eb446f9c968 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.469213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1345/2698] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:06.473178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee183e7-510c-442a-bf00-3eb446f9c968 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.484449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee183e7-510c-442a-bf00-3eb446f9c968 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.489678 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb6f28-ff5d-44bb-82ca-236c5bf819db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.490615 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb6f28-ff5d-44bb-82ca-236c5bf819db admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.490900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb6f28-ff5d-44bb-82ca-236c5bf819db admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.491178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb6f28-ff5d-44bb-82ca-236c5bf819db admin admin] PATH_INFO: `/v3/projects/0f2c1dc068f24e4096090a140a1b9561/users/989a23b9cf9b4777a1cefb9221453792/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.507051 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb6f28-ff5d-44bb-82ca-236c5bf819db admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2c1dc068f24e4096090a140a1b9561, user_id=989a23b9cf9b4777a1cefb9221453792, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.509849 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb6f28-ff5d-44bb-82ca-236c5bf819db admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.519953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1354/2699] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:06 2022] POST /identity/v3/projects => generated 373 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:06.524590 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76066c14-5e54-46dd-8305-b2c6de04e65c admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.590075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76066c14-5e54-46dd-8305-b2c6de04e65c admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.594864 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faac47f-d0ae-4077-b038-39370044ef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.595580 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac47f-d0ae-4077-b038-39370044ef7b admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.596009 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac47f-d0ae-4077-b038-39370044ef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.596288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac47f-d0ae-4077-b038-39370044ef7b admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.596985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faac47f-d0ae-4077-b038-39370044ef7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.598935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faac47f-d0ae-4077-b038-39370044ef7b admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.661807 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:06.668881 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1346/2700] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:06 2022] PUT /identity/v3/projects/0f2c1dc068f24e4096090a140a1b9561/users/989a23b9cf9b4777a1cefb9221453792/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:06.680218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d0149-e200-4340-abe6-d314038608d2 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.680552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d0149-e200-4340-abe6-d314038608d2 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.680865 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d0149-e200-4340-abe6-d314038608d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.708035 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-c49d0149-e200-4340-abe6-d314038608d2 None None] MFA Rules not processed for user `989a23b9cf9b4777a1cefb9221453792`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:06.709359 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c49d0149-e200-4340-abe6-d314038608d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:06.739622 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1355/2701] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:06 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:06.743987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faac47f-d0ae-4077-b038-39370044ef7b admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.754088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faac47f-d0ae-4077-b038-39370044ef7b admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.784745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1347/2702] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:06.790205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d0149-e200-4340-abe6-d314038608d2 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.824639 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b64d3-de66-41a3-a842-4d2db61a237d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.825582 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b64d3-de66-41a3-a842-4d2db61a237d admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.826197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b64d3-de66-41a3-a842-4d2db61a237d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.826679 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b64d3-de66-41a3-a842-4d2db61a237d admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.828207 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b64d3-de66-41a3-a842-4d2db61a237d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.830336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b64d3-de66-41a3-a842-4d2db61a237d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.835240 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:06.859708 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1356/2703] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:06.861008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b64d3-de66-41a3-a842-4d2db61a237d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.871437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b64d3-de66-41a3-a842-4d2db61a237d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.876868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5f672-9acb-4cff-a762-528ecfeecb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.877862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5f672-9acb-4cff-a762-528ecfeecb89 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.878112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5f672-9acb-4cff-a762-528ecfeecb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.878465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5f672-9acb-4cff-a762-528ecfeecb89 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.879390 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5f672-9acb-4cff-a762-528ecfeecb89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.881269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5f672-9acb-4cff-a762-528ecfeecb89 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.886294 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d0149-e200-4340-abe6-d314038608d2 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.887993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1e21e-021c-412d-bfd6-99b95890cc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.888805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1e21e-021c-412d-bfd6-99b95890cc13 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.889167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1e21e-021c-412d-bfd6-99b95890cc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.889518 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1e21e-021c-412d-bfd6-99b95890cc13 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.890324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1e21e-021c-412d-bfd6-99b95890cc13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.892399 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1e21e-021c-412d-bfd6-99b95890cc13 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:06.902854 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1357/2704] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:06.904683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5f672-9acb-4cff-a762-528ecfeecb89 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:06.915627 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5f672-9acb-4cff-a762-528ecfeecb89 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:06.927007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824206f-d54c-4d3c-86a2-d336d64f2fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:06.927007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824206f-d54c-4d3c-86a2-d336d64f2fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:06.927007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824206f-d54c-4d3c-86a2-d336d64f2fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:06.927007 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824206f-d54c-4d3c-86a2-d336d64f2fe6 admin admin] PATH_INFO: `/v3/projects/1a14d18943b64d8a914952bd1c0e05e1/users/b6e8372460404eef934cd8b6bc7a290f/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:06.938953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824206f-d54c-4d3c-86a2-d336d64f2fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a14d18943b64d8a914952bd1c0e05e1, user_id=b6e8372460404eef934cd8b6bc7a290f, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:06.941851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824206f-d54c-4d3c-86a2-d336d64f2fe6 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:07.041767 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1348/2705] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:06 2022] POST /identity/v3/projects => generated 385 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:07.045993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1e21e-021c-412d-bfd6-99b95890cc13 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:07.078608 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1358/2706] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:06 2022] PUT /identity/v3/projects/1a14d18943b64d8a914952bd1c0e05e1/users/b6e8372460404eef934cd8b6bc7a290f/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:07.083780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806179ca-dbd7-4ff4-be5f-6b8da46b7b37 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.084075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806179ca-dbd7-4ff4-be5f-6b8da46b7b37 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.084360 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806179ca-dbd7-4ff4-be5f-6b8da46b7b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.104785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-806179ca-dbd7-4ff4-be5f-6b8da46b7b37 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:07.111555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-806179ca-dbd7-4ff4-be5f-6b8da46b7b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:07.118957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1e21e-021c-412d-bfd6-99b95890cc13 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:07.177314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1359/2707] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:07.186290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc4548-5b54-4cc5-8248-d9a355ce0716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:07.186972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4548-5b54-4cc5-8248-d9a355ce0716 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.187242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806179ca-dbd7-4ff4-be5f-6b8da46b7b37 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:07.187391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4548-5b54-4cc5-8248-d9a355ce0716 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.187722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4548-5b54-4cc5-8248-d9a355ce0716 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.188461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4548-5b54-4cc5-8248-d9a355ce0716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:07.188962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-806179ca-dbd7-4ff4-be5f-6b8da46b7b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:07.190527 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4548-5b54-4cc5-8248-d9a355ce0716 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:07.250977 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806179ca-dbd7-4ff4-be5f-6b8da46b7b37 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:07.254901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded046f-cecd-4e12-839d-b70f7cd8fe11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:07.255549 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded046f-cecd-4e12-839d-b70f7cd8fe11 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.255816 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded046f-cecd-4e12-839d-b70f7cd8fe11 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.256090 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded046f-cecd-4e12-839d-b70f7cd8fe11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.266336 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded046f-cecd-4e12-839d-b70f7cd8fe11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:07.270032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded046f-cecd-4e12-839d-b70f7cd8fe11 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:07.278945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1360/2708] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:04:07.311135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1349/2709] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:07 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:07.311373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded046f-cecd-4e12-839d-b70f7cd8fe11 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:07.322484 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded046f-cecd-4e12-839d-b70f7cd8fe11 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:07.328208 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cde9f-bae5-425b-93b8-67608acf7f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:07.329010 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cde9f-bae5-425b-93b8-67608acf7f27 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.329276 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cde9f-bae5-425b-93b8-67608acf7f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.329561 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cde9f-bae5-425b-93b8-67608acf7f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.330332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cde9f-bae5-425b-93b8-67608acf7f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:07.332248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cde9f-bae5-425b-93b8-67608acf7f27 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:07.343689 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1361/2710] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:07.347805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc4548-5b54-4cc5-8248-d9a355ce0716 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:07.358503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc4548-5b54-4cc5-8248-d9a355ce0716 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:07.363428 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbfbf6-6cc7-47bc-a64b-23e4573bc382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:07.364099 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbfbf6-6cc7-47bc-a64b-23e4573bc382 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.364389 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbfbf6-6cc7-47bc-a64b-23e4573bc382 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.364673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbfbf6-6cc7-47bc-a64b-23e4573bc382 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.365423 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbfbf6-6cc7-47bc-a64b-23e4573bc382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:07.367083 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbfbf6-6cc7-47bc-a64b-23e4573bc382 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:07.379867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1350/2711] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:07.383280 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cde9f-bae5-425b-93b8-67608acf7f27 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:07.394632 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cde9f-bae5-425b-93b8-67608acf7f27 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:07.399650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64276f-0eb0-4583-bce3-2ee621fab3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:07.400544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64276f-0eb0-4583-bce3-2ee621fab3ee admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.400825 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64276f-0eb0-4583-bce3-2ee621fab3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.401102 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64276f-0eb0-4583-bce3-2ee621fab3ee admin admin] PATH_INFO: `/v3/projects/4bc178f725dd4aabb2f0f0033f1dc62f/users/78e84e5318c3485a966900ff95b782b2/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.417283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64276f-0eb0-4583-bce3-2ee621fab3ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc178f725dd4aabb2f0f0033f1dc62f, user_id=78e84e5318c3485a966900ff95b782b2, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:07.420382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64276f-0eb0-4583-bce3-2ee621fab3ee admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:07.569256 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:07.606950 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba373e97-2947-4b10-8a20-d43ac699e629 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.607328 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba373e97-2947-4b10-8a20-d43ac699e629 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.607612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba373e97-2947-4b10-8a20-d43ac699e629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.624034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1362/2712] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:07 2022] PUT /identity/v3/projects/4bc178f725dd4aabb2f0f0033f1dc62f/users/78e84e5318c3485a966900ff95b782b2/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:07.643675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ba373e97-2947-4b10-8a20-d43ac699e629 None None] MFA Rules not processed for user `613c0627a67041cd858b61ecc3d08421`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:07.645053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ba373e97-2947-4b10-8a20-d43ac699e629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:07.690328 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:07.693245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ba0f-8a79-4fc7-91fb-6384240af885 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.693245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ba0f-8a79-4fc7-91fb-6384240af885 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.693499 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ba0f-8a79-4fc7-91fb-6384240af885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.708593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1351/2713] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 14 16:04:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:07.725711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-6559ba0f-8a79-4fc7-91fb-6384240af885 None None] MFA Rules not processed for user `78e84e5318c3485a966900ff95b782b2`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:07.727465 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6559ba0f-8a79-4fc7-91fb-6384240af885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:07.810211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1363/2714] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:07.815395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba373e97-2947-4b10-8a20-d43ac699e629 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:07.864571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ba0f-8a79-4fc7-91fb-6384240af885 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:07.866157 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-6559ba0f-8a79-4fc7-91fb-6384240af885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:07.889510 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba373e97-2947-4b10-8a20-d43ac699e629 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:07.895481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92005a1-653a-40e0-8361-7f27d448bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:07.895481 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92005a1-653a-40e0-8361-7f27d448bb31 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.895762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92005a1-653a-40e0-8361-7f27d448bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.896036 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92005a1-653a-40e0-8361-7f27d448bb31 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.896750 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92005a1-653a-40e0-8361-7f27d448bb31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:07.898733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92005a1-653a-40e0-8361-7f27d448bb31 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:07.947732 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ba0f-8a79-4fc7-91fb-6384240af885 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:07.954023 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a5cea-cdec-4665-9a2b-5deee9d6b5ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f36b4517ee4b64bc43e000c854df30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:07.954704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a5cea-cdec-4665-9a2b-5deee9d6b5ac service swift] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:07.954986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a5cea-cdec-4665-9a2b-5deee9d6b5ac service swift] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:07.955260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a5cea-cdec-4665-9a2b-5deee9d6b5ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:07.965777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a5cea-cdec-4665-9a2b-5deee9d6b5ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:07.968219 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a5cea-cdec-4665-9a2b-5deee9d6b5ac service swift] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:07.977931 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1364/2715] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 14 16:04:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 16:04:08.016161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1352/2716] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:07 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:08.017332 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a5cea-cdec-4665-9a2b-5deee9d6b5ac service swift] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:08.031353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a5cea-cdec-4665-9a2b-5deee9d6b5ac service swift] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:08.111260 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35227eee-b112-4e4c-a9d4-dbf1bb38de46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:08.112104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35227eee-b112-4e4c-a9d4-dbf1bb38de46 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:08.112392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35227eee-b112-4e4c-a9d4-dbf1bb38de46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:08.112671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35227eee-b112-4e4c-a9d4-dbf1bb38de46 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:08.113517 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35227eee-b112-4e4c-a9d4-dbf1bb38de46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:08.115534 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35227eee-b112-4e4c-a9d4-dbf1bb38de46 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:08.202640 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1365/2717] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:08 2022] POST /identity/v3/users => generated 299 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:08.206856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92005a1-653a-40e0-8361-7f27d448bb31 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:08.217657 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92005a1-653a-40e0-8361-7f27d448bb31 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:08.226699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf29742-0a75-4b9a-a0f1-c9f61e3f130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:08.226699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf29742-0a75-4b9a-a0f1-c9f61e3f130b admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:08.226699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf29742-0a75-4b9a-a0f1-c9f61e3f130b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:08.226699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf29742-0a75-4b9a-a0f1-c9f61e3f130b admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:08.227113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf29742-0a75-4b9a-a0f1-c9f61e3f130b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:08.227113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf29742-0a75-4b9a-a0f1-c9f61e3f130b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:08.238193 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1353/2718] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:08.243612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35227eee-b112-4e4c-a9d4-dbf1bb38de46 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:08.255450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:08.256855 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35227eee-b112-4e4c-a9d4-dbf1bb38de46 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:08.263031 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b507fae-8212-4823-b63a-b9b2ddce2374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:08.264502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b507fae-8212-4823-b63a-b9b2ddce2374 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:08.264786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b507fae-8212-4823-b63a-b9b2ddce2374 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:08.265058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b507fae-8212-4823-b63a-b9b2ddce2374 admin admin] PATH_INFO: `/v3/projects/db845f0a44f24d43a205fa8777e0fe9b/users/347359a24943454782d7e58a3c0492c4/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:08.281626 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b507fae-8212-4823-b63a-b9b2ddce2374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db845f0a44f24d43a205fa8777e0fe9b, user_id=347359a24943454782d7e58a3c0492c4, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:08.284460 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b507fae-8212-4823-b63a-b9b2ddce2374 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:08.358012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:08.419795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1366/2719] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:08 2022] PUT /identity/v3/projects/db845f0a44f24d43a205fa8777e0fe9b/users/347359a24943454782d7e58a3c0492c4/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:08.424509 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf29742-0a75-4b9a-a0f1-c9f61e3f130b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:08.436016 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf29742-0a75-4b9a-a0f1-c9f61e3f130b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:08.497414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b94a56-e7c9-4b47-b1ad-b9ad3b0aae46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:08.498681 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b94a56-e7c9-4b47-b1ad-b9ad3b0aae46 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:08.498970 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b94a56-e7c9-4b47-b1ad-b9ad3b0aae46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:08.499242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b94a56-e7c9-4b47-b1ad-b9ad3b0aae46 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:08.500014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b94a56-e7c9-4b47-b1ad-b9ad3b0aae46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:08.501620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b94a56-e7c9-4b47-b1ad-b9ad3b0aae46 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:08.512553 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1354/2720] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:08.516897 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b507fae-8212-4823-b63a-b9b2ddce2374 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:08.527947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b507fae-8212-4823-b63a-b9b2ddce2374 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:08.533236 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fd61b-d85e-4f8c-a399-d0d060d4159d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:08.534191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fd61b-d85e-4f8c-a399-d0d060d4159d admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:08.534504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fd61b-d85e-4f8c-a399-d0d060d4159d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:08.534750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fd61b-d85e-4f8c-a399-d0d060d4159d admin admin] PATH_INFO: `/v3/projects/db845f0a44f24d43a205fa8777e0fe9b/users/347359a24943454782d7e58a3c0492c4/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:08.536787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fd61b-d85e-4f8c-a399-d0d060d4159d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db845f0a44f24d43a205fa8777e0fe9b, user_id=347359a24943454782d7e58a3c0492c4, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:08.540444 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fd61b-d85e-4f8c-a399-d0d060d4159d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:08.554245 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1367/2721] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:08 2022] PUT /identity/v3/projects/db845f0a44f24d43a205fa8777e0fe9b/users/347359a24943454782d7e58a3c0492c4/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:08.565094 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d912d-39fb-4f48-ab67-4e200753e9ab None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:08.565373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d912d-39fb-4f48-ab67-4e200753e9ab None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:08.565659 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d912d-39fb-4f48-ab67-4e200753e9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:08.604208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-da0d912d-39fb-4f48-ab67-4e200753e9ab None None] MFA Rules not processed for user `347359a24943454782d7e58a3c0492c4`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:08.605552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-da0d912d-39fb-4f48-ab67-4e200753e9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:08.682093 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1355/2722] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:08.696681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fd61b-d85e-4f8c-a399-d0d060d4159d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:08.698936 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-592fd61b-d85e-4f8c-a399-d0d060d4159d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:08.711742 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:08.759046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fd61b-d85e-4f8c-a399-d0d060d4159d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:08.762880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ff52d-d8bd-419e-ac2f-8b5346ec0132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:08.763606 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ff52d-d8bd-419e-ac2f-8b5346ec0132 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:08.763895 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ff52d-d8bd-419e-ac2f-8b5346ec0132 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:08.764176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ff52d-d8bd-419e-ac2f-8b5346ec0132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:08.833487 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ff52d-d8bd-419e-ac2f-8b5346ec0132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:08.835016 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ff52d-d8bd-419e-ac2f-8b5346ec0132 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:08.845087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1368/2723] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:08.903485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:08.952927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18b3cf-78db-4883-8a1f-f2475eb37a37 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:08.953224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18b3cf-78db-4883-8a1f-f2475eb37a37 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:08.955220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18b3cf-78db-4883-8a1f-f2475eb37a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:09.008809 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5f18b3cf-78db-4883-8a1f-f2475eb37a37 None None] MFA Rules not processed for user `b6e8372460404eef934cd8b6bc7a290f`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:09.010311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5f18b3cf-78db-4883-8a1f-f2475eb37a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:09.095239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1356/2724] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:09.099787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ff52d-d8bd-419e-ac2f-8b5346ec0132 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:09.164827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ff52d-d8bd-419e-ac2f-8b5346ec0132 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:09.170412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa93de-2494-4df3-9999-c0c7c6e75f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:09.171308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa93de-2494-4df3-9999-c0c7c6e75f60 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:09.171851 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa93de-2494-4df3-9999-c0c7c6e75f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:09.172146 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa93de-2494-4df3-9999-c0c7c6e75f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:09.172835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa93de-2494-4df3-9999-c0c7c6e75f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:09.174941 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa93de-2494-4df3-9999-c0c7c6e75f60 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:09.353990 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1369/2725] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:09 2022] POST /identity/v3/projects => generated 371 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:09.358270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18b3cf-78db-4883-8a1f-f2475eb37a37 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:09.369693 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18b3cf-78db-4883-8a1f-f2475eb37a37 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:09.458780 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8a3c5-f9f6-4277-a81c-b801c072e61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:09.459536 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8a3c5-f9f6-4277-a81c-b801c072e61e admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:09.459863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8a3c5-f9f6-4277-a81c-b801c072e61e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:09.460182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd8a3c5-f9f6-4277-a81c-b801c072e61e admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:09.460974 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8a3c5-f9f6-4277-a81c-b801c072e61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:09.464263 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd8a3c5-f9f6-4277-a81c-b801c072e61e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:09.615682 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1357/2726] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:09 2022] POST /identity/v3/users => generated 292 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:09.619919 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa93de-2494-4df3-9999-c0c7c6e75f60 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:09.631333 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa93de-2494-4df3-9999-c0c7c6e75f60 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:09.639337 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c6169-8f33-4029-ae3b-c87cd76d00df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:09.640068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c6169-8f33-4029-ae3b-c87cd76d00df admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:09.640347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c6169-8f33-4029-ae3b-c87cd76d00df admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:09.640625 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c6169-8f33-4029-ae3b-c87cd76d00df admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:09.641453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c6169-8f33-4029-ae3b-c87cd76d00df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:09.643068 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c6169-8f33-4029-ae3b-c87cd76d00df admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:09.655506 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1370/2727] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:09.660025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8a3c5-f9f6-4277-a81c-b801c072e61e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:09.673012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd8a3c5-f9f6-4277-a81c-b801c072e61e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:09.679174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fba32a-1c85-44fe-8520-69d50f227fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:09.684107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fba32a-1c85-44fe-8520-69d50f227fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:09.684107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fba32a-1c85-44fe-8520-69d50f227fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:09.684107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fba32a-1c85-44fe-8520-69d50f227fd0 admin admin] PATH_INFO: `/v3/projects/1021cb5b8d5843bc85cf553692a7b2e8/users/002642649f854d2a8fab486e3d7f8883/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:09.700401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fba32a-1c85-44fe-8520-69d50f227fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1021cb5b8d5843bc85cf553692a7b2e8, user_id=002642649f854d2a8fab486e3d7f8883, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:09.703419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fba32a-1c85-44fe-8520-69d50f227fd0 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:09.710452 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:09.792745 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1358/2728] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:09 2022] PUT /identity/v3/projects/1021cb5b8d5843bc85cf553692a7b2e8/users/002642649f854d2a8fab486e3d7f8883/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:09.797736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c6169-8f33-4029-ae3b-c87cd76d00df admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:09.808267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c6169-8f33-4029-ae3b-c87cd76d00df admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:09.871438 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:09.886878 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18af067-b202-4824-8443-31be08018138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:09.890776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18af067-b202-4824-8443-31be08018138 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:09.890776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18af067-b202-4824-8443-31be08018138 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:09.890776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18af067-b202-4824-8443-31be08018138 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:09.890776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18af067-b202-4824-8443-31be08018138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:09.890776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18af067-b202-4824-8443-31be08018138 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:09.907353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1371/2729] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:09.909112 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fba32a-1c85-44fe-8520-69d50f227fd0 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:09.926706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fba32a-1c85-44fe-8520-69d50f227fd0 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:09.932034 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d339c-a667-445b-814b-3f5e1a7d5374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:09.933061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d339c-a667-445b-814b-3f5e1a7d5374 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:09.933475 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d339c-a667-445b-814b-3f5e1a7d5374 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:09.933912 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d339c-a667-445b-814b-3f5e1a7d5374 admin admin] PATH_INFO: `/v3/projects/1021cb5b8d5843bc85cf553692a7b2e8/users/002642649f854d2a8fab486e3d7f8883/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:09.936050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d339c-a667-445b-814b-3f5e1a7d5374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1021cb5b8d5843bc85cf553692a7b2e8, user_id=002642649f854d2a8fab486e3d7f8883, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:09.939305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d339c-a667-445b-814b-3f5e1a7d5374 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:09.960386 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1359/2730] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:09 2022] PUT /identity/v3/projects/1021cb5b8d5843bc85cf553692a7b2e8/users/002642649f854d2a8fab486e3d7f8883/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:11.273149 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:11.300391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:11.930213 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b36dd-7a4c-465c-b447-679736871416 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:11.930504 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b36dd-7a4c-465c-b447-679736871416 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:11.930786 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b36dd-7a4c-465c-b447-679736871416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:11.964935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-777b36dd-7a4c-465c-b447-679736871416 None None] MFA Rules not processed for user `002642649f854d2a8fab486e3d7f8883`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:11.964935 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-777b36dd-7a4c-465c-b447-679736871416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:12.055993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1372/2731] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:11 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:12.068340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d339c-a667-445b-814b-3f5e1a7d5374 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:12.185582 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d339c-a667-445b-814b-3f5e1a7d5374 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:12.189307 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5df5-8453-4ab5-8377-bc62e35cf7cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:12.190060 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5df5-8453-4ab5-8377-bc62e35cf7cd service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:12.191262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5df5-8453-4ab5-8377-bc62e35cf7cd service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:12.191739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5df5-8453-4ab5-8377-bc62e35cf7cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:12.301867 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b5df5-8453-4ab5-8377-bc62e35cf7cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:12.303566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b5df5-8453-4ab5-8377-bc62e35cf7cd service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:12.316537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1360/2732] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:12.411269 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b36dd-7a4c-465c-b447-679736871416 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:12.412754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-777b36dd-7a4c-465c-b447-679736871416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:12.430015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b36dd-7a4c-465c-b447-679736871416 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:12.434514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e70383-20e3-4cda-8a22-257a79b16621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:12.435838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e70383-20e3-4cda-8a22-257a79b16621 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:12.436196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e70383-20e3-4cda-8a22-257a79b16621 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:12.436724 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e70383-20e3-4cda-8a22-257a79b16621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:12.505114 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e70383-20e3-4cda-8a22-257a79b16621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:12.506723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e70383-20e3-4cda-8a22-257a79b16621 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:12.519109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1373/2733] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:12.821559 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:14.124760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:14.491224 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5df5-8453-4ab5-8377-bc62e35cf7cd service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:14.499959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5df5-8453-4ab5-8377-bc62e35cf7cd service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:14.504291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4abe1-af4f-4772-9dd2-c41cb3356e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:14.505119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4abe1-af4f-4772-9dd2-c41cb3356e93 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:14.505408 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4abe1-af4f-4772-9dd2-c41cb3356e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:14.505928 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4abe1-af4f-4772-9dd2-c41cb3356e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:14.516191 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4abe1-af4f-4772-9dd2-c41cb3356e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:14.517630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4abe1-af4f-4772-9dd2-c41cb3356e93 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:14.528479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1361/2734] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:14.790423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e70383-20e3-4cda-8a22-257a79b16621 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:14.804414 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e70383-20e3-4cda-8a22-257a79b16621 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:14.824795 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dfd05-6918-4589-a1b2-c58e8c71e440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:14.825642 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dfd05-6918-4589-a1b2-c58e8c71e440 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:14.826548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dfd05-6918-4589-a1b2-c58e8c71e440 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:14.826899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dfd05-6918-4589-a1b2-c58e8c71e440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:14.862689 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:14.925959 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633dfd05-6918-4589-a1b2-c58e8c71e440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:14.927737 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633dfd05-6918-4589-a1b2-c58e8c71e440 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:14.944025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1374/2735] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:18.054341 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4abe1-af4f-4772-9dd2-c41cb3356e93 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:18.056576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-62c4abe1-af4f-4772-9dd2-c41cb3356e93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:18.134433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4abe1-af4f-4772-9dd2-c41cb3356e93 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:18.139734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:18.140641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:18.140951 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:18.141273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] PATH_INFO: `/v3/users/2c1f1c69e35847b9a6323d8dd3863bf7` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:18.142574 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c1f1c69e35847b9a6323d8dd3863bf7)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:18.144556 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:18.332926 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c1f1c69e35847b9a6323d8dd3863bf7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:18.487393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c1f1c69e35847b9a6323d8dd3863bf7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:18.488022 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c1f1c69e35847b9a6323d8dd3863bf7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:18.497629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:18.500074 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c1f1c69e35847b9a6323d8dd3863bf7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:18.500761 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c1f1c69e35847b9a6323d8dd3863bf7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:18.521474 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1362/2736] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:18 2022] DELETE /identity/v3/users/2c1f1c69e35847b9a6323d8dd3863bf7 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:18.729206 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dfd05-6918-4589-a1b2-c58e8c71e440 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:18.730697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-633dfd05-6918-4589-a1b2-c58e8c71e440 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:18.744392 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dfd05-6918-4589-a1b2-c58e8c71e440 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:18.807452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:18.810325 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:18.854192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:18.854192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:18.854192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:18.854192 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] PATH_INFO: `/v3/users/eaf17ff6d01c48b58de3c44d83fdab1a` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:18.854536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf17ff6d01c48b58de3c44d83fdab1a)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:18.854536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:18.903368 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb2873-3d82-42a4-a5bd-f45ece1e206e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:18.908610 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:18.909538 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:18.909851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:18.910161 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] PATH_INFO: `/v3/projects/923fdefb5a7246038c012ee78bf00dc2` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:18.911577 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=923fdefb5a7246038c012ee78bf00dc2)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:18.913714 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:19.171928 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf17ff6d01c48b58de3c44d83fdab1a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.172172 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '923fdefb5a7246038c012ee78bf00dc2', 'request_id': 'req-097c8ba8-110f-48f9-b5de-39056e7430d1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.177947 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '923fdefb5a7246038c012ee78bf00dc2', 'request_id': 'req-097c8ba8-110f-48f9-b5de-39056e7430d1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.292685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf17ff6d01c48b58de3c44d83fdab1a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.293300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf17ff6d01c48b58de3c44d83fdab1a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.308308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf17ff6d01c48b58de3c44d83fdab1a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.308308 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-8f1f3446-4c86-468d-be7f-51a5fbf19609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf17ff6d01c48b58de3c44d83fdab1a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.320650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1375/2737] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:18 2022] DELETE /identity/v3/users/eaf17ff6d01c48b58de3c44d83fdab1a => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:19.324726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc7f78-de14-40b9-90dc-1f90b7ea5e15 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:19.325022 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc7f78-de14-40b9-90dc-1f90b7ea5e15 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:19.325306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc7f78-de14-40b9-90dc-1f90b7ea5e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:19.341796 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-92fc7f78-de14-40b9-90dc-1f90b7ea5e15 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:19.343730 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-92fc7f78-de14-40b9-90dc-1f90b7ea5e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:19.373250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:19.396863 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '923fdefb5a7246038c012ee78bf00dc2', 'request_id': 'req-097c8ba8-110f-48f9-b5de-39056e7430d1'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.410755 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1376/2738] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:19.411461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] The token cache is being invalidate because project 923fdefb5a7246038c012ee78bf00dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:19.411763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.412552 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.417208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1363/2739] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:18 2022] DELETE /identity/v3/projects/923fdefb5a7246038c012ee78bf00dc2 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:19.418763 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc7f78-de14-40b9-90dc-1f90b7ea5e15 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:19.420656 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-92fc7f78-de14-40b9-90dc-1f90b7ea5e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:19.422123 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:19.423480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:19.435140 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc7f78-de14-40b9-90dc-1f90b7ea5e15 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:19.441272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:19.442296 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:19.442637 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:19.443129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] PATH_INFO: `/v3/users/47ea9a9b8739445183e53ade0278dd17` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:19.444613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ea9a9b8739445183e53ade0278dd17)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:19.447158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:19.491860 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c8ba8-110f-48f9-b5de-39056e7430d1 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:19.513110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:19.543239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:19.543892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:19.544177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:19.544461 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:19.547053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:19.558580 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:19.559955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:19.569957 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1364/2740] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:04:19.784277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ea9a9b8739445183e53ade0278dd17'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.900066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ea9a9b8739445183e53ade0278dd17'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.901037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ea9a9b8739445183e53ade0278dd17'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.915880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ea9a9b8739445183e53ade0278dd17'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.915880 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ea9a9b8739445183e53ade0278dd17'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:19.917067 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:19.918608 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:19.931418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22b17a-ea5a-4ca9-9cf0-b083ff9e2c01 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:19.932136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1377/2741] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:19 2022] DELETE /identity/v3/users/47ea9a9b8739445183e53ade0278dd17 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:19.997921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:19.999367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:19.999660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.001218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] PATH_INFO: `/v3/users/989a23b9cf9b4777a1cefb9221453792` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.001420 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=989a23b9cf9b4777a1cefb9221453792)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:20.003220 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:20.068172 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:20.070171 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:20.084230 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07141ba4-2c86-4e35-a9e2-16abca67c0e5 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:20.089925 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:20.092986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:20.092986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.092986 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] PATH_INFO: `/v3/projects/59b915690792451f8519082c4480009d` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.093412 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b915690792451f8519082c4480009d)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:20.095902 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:20.227664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '989a23b9cf9b4777a1cefb9221453792'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.258739 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b915690792451f8519082c4480009d', 'request_id': 'req-45063690-e961-4257-ae84-d40f688ab6e4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.265211 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b915690792451f8519082c4480009d', 'request_id': 'req-45063690-e961-4257-ae84-d40f688ab6e4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.300126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '989a23b9cf9b4777a1cefb9221453792'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.300823 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '989a23b9cf9b4777a1cefb9221453792'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.308986 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '989a23b9cf9b4777a1cefb9221453792'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.309607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '989a23b9cf9b4777a1cefb9221453792'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.328630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1365/2742] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:19 2022] DELETE /identity/v3/users/989a23b9cf9b4777a1cefb9221453792 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:20.329626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:20.331088 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:20.396154 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eda9ee-7aeb-413d-aaea-57116332d630 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:20.400743 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:20.401620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:20.401958 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.402243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] PATH_INFO: `/v3/projects/bf058108de93470289cd511e2b038cd0` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.403456 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf058108de93470289cd511e2b038cd0)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:20.405384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:20.408538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b915690792451f8519082c4480009d', 'request_id': 'req-45063690-e961-4257-ae84-d40f688ab6e4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.425026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] The token cache is being invalidate because project 59b915690792451f8519082c4480009d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:20.425324 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.425572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:20.426056 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-45063690-e961-4257-ae84-d40f688ab6e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.427668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1378/2743] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:20 2022] DELETE /identity/v3/projects/59b915690792451f8519082c4480009d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:20.428987 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca7014-728b-4c51-a5c3-dfb6abe65c34 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:20.429275 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca7014-728b-4c51-a5c3-dfb6abe65c34 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.429767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca7014-728b-4c51-a5c3-dfb6abe65c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.448047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c0ca7014-728b-4c51-a5c3-dfb6abe65c34 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:20.456396 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c0ca7014-728b-4c51-a5c3-dfb6abe65c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:20.456918 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1379/2744] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:20.458247 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eafc1-2ff3-469e-ad67-b91566dba216 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:20.458533 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eafc1-2ff3-469e-ad67-b91566dba216 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.458812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189eafc1-2ff3-469e-ad67-b91566dba216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.476702 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-189eafc1-2ff3-469e-ad67-b91566dba216 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:20.477934 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-189eafc1-2ff3-469e-ad67-b91566dba216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:20.486386 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1380/2745] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:20.487175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189eafc1-2ff3-469e-ad67-b91566dba216 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:20.490701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-189eafc1-2ff3-469e-ad67-b91566dba216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:20.508944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:20.548118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189eafc1-2ff3-469e-ad67-b91566dba216 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:20.552201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab364bc-bb83-4ade-ad6c-41d7a9a62561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:20.553015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab364bc-bb83-4ade-ad6c-41d7a9a62561 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:20.553374 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab364bc-bb83-4ade-ad6c-41d7a9a62561 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.553723 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab364bc-bb83-4ade-ad6c-41d7a9a62561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.564366 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab364bc-bb83-4ade-ad6c-41d7a9a62561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:20.565771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab364bc-bb83-4ade-ad6c-41d7a9a62561 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:20.571004 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf058108de93470289cd511e2b038cd0', 'request_id': 'req-5a485fdc-ceed-40b4-b7da-d7b082743bb7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.575553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1381/2746] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:20.576934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf058108de93470289cd511e2b038cd0', 'request_id': 'req-5a485fdc-ceed-40b4-b7da-d7b082743bb7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.577075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4bbc-66b8-4424-8c63-c6858f15a278 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:20.577268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4bbc-66b8-4424-8c63-c6858f15a278 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.577548 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4bbc-66b8-4424-8c63-c6858f15a278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.597711 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-cffa4bbc-66b8-4424-8c63-c6858f15a278 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:20.598996 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-cffa4bbc-66b8-4424-8c63-c6858f15a278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:20.661288 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1382/2747] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:20.665310 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4bbc-66b8-4424-8c63-c6858f15a278 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:20.672380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf058108de93470289cd511e2b038cd0', 'request_id': 'req-5a485fdc-ceed-40b4-b7da-d7b082743bb7'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.676156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4bbc-66b8-4424-8c63-c6858f15a278 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:20.680813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e098b-28bc-4e70-b4ab-220f2a0973b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:20.681490 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e098b-28bc-4e70-b4ab-220f2a0973b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:20.681780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e098b-28bc-4e70-b4ab-220f2a0973b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.682099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e098b-28bc-4e70-b4ab-220f2a0973b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.682835 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e098b-28bc-4e70-b4ab-220f2a0973b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:20.683151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e098b-28bc-4e70-b4ab-220f2a0973b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:20.684785 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e098b-28bc-4e70-b4ab-220f2a0973b1 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:20.685119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] The token cache is being invalidate because project bf058108de93470289cd511e2b038cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:20.685415 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.686754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:20.688335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1366/2748] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:20 2022] DELETE /identity/v3/projects/bf058108de93470289cd511e2b038cd0 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:20.697964 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1383/2749] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:04:20.703251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:20.705184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:20.720562 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a485fdc-ceed-40b4-b7da-d7b082743bb7 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:20.725658 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5464e2b-608a-4a9d-ad1b-0df89e776742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:20.726432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5464e2b-608a-4a9d-ad1b-0df89e776742 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:20.726942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5464e2b-608a-4a9d-ad1b-0df89e776742 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.727234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5464e2b-608a-4a9d-ad1b-0df89e776742 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.727968 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5464e2b-608a-4a9d-ad1b-0df89e776742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:20.730141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5464e2b-608a-4a9d-ad1b-0df89e776742 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:20.897725 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1367/2750] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:20 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:20.902119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e098b-28bc-4e70-b4ab-220f2a0973b1 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:20.914379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e098b-28bc-4e70-b4ab-220f2a0973b1 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:20.985540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010be81-4c08-41d1-adb7-e19090cce228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:20.985540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010be81-4c08-41d1-adb7-e19090cce228 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:20.985540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010be81-4c08-41d1-adb7-e19090cce228 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:20.985540 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010be81-4c08-41d1-adb7-e19090cce228 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:20.986232 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010be81-4c08-41d1-adb7-e19090cce228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:20.988177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010be81-4c08-41d1-adb7-e19090cce228 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:21.074574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1384/2751] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:20 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:21.076286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5464e2b-608a-4a9d-ad1b-0df89e776742 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:21.085053 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010be81-4c08-41d1-adb7-e19090cce228 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:21.088422 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-3010be81-4c08-41d1-adb7-e19090cce228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:21.092753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5464e2b-608a-4a9d-ad1b-0df89e776742 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:21.098496 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac62fcb-5795-4c06-be74-013d9d97c27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:21.099302 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac62fcb-5795-4c06-be74-013d9d97c27a admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:21.099648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac62fcb-5795-4c06-be74-013d9d97c27a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:21.104828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac62fcb-5795-4c06-be74-013d9d97c27a admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:21.104828 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac62fcb-5795-4c06-be74-013d9d97c27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:21.105665 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010be81-4c08-41d1-adb7-e19090cce228 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:21.105776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac62fcb-5795-4c06-be74-013d9d97c27a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:21.108634 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:21.109635 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:21.110131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:21.110559 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] PATH_INFO: `/v3/projects/0f2c1dc068f24e4096090a140a1b9561` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:21.112000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f2c1dc068f24e4096090a140a1b9561)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:21.114097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:21.117572 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1368/2752] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:21.121584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac62fcb-5795-4c06-be74-013d9d97c27a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:21.134320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac62fcb-5795-4c06-be74-013d9d97c27a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:21.144929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5905402-bfa2-4173-b0a8-6c2a36a2925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:21.144929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5905402-bfa2-4173-b0a8-6c2a36a2925e admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:21.144929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5905402-bfa2-4173-b0a8-6c2a36a2925e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:21.144929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5905402-bfa2-4173-b0a8-6c2a36a2925e admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:21.145439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5905402-bfa2-4173-b0a8-6c2a36a2925e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:21.145439 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5905402-bfa2-4173-b0a8-6c2a36a2925e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:21.153479 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1369/2753] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:21.157687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5905402-bfa2-4173-b0a8-6c2a36a2925e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:21.167609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5905402-bfa2-4173-b0a8-6c2a36a2925e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:21.173692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af38e09-68b7-4547-a621-026192532e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:21.174330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af38e09-68b7-4547-a621-026192532e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:21.174330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af38e09-68b7-4547-a621-026192532e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:21.174513 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af38e09-68b7-4547-a621-026192532e0f admin admin] PATH_INFO: `/v3/projects/31d5c1320c1646769d9b4d7322c0a6ee/users/bee9fe3603134ce0ab3fbeb8ca4877ea/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:21.190739 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af38e09-68b7-4547-a621-026192532e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d5c1320c1646769d9b4d7322c0a6ee, user_id=bee9fe3603134ce0ab3fbeb8ca4877ea, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:21.193596 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af38e09-68b7-4547-a621-026192532e0f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:21.357942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1370/2754] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:21 2022] PUT /identity/v3/projects/31d5c1320c1646769d9b4d7322c0a6ee/users/bee9fe3603134ce0ab3fbeb8ca4877ea/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:21.368097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f2c1dc068f24e4096090a140a1b9561', 'request_id': 'req-1ab763ca-423c-457c-b50d-3dd674732a8a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.369254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9d1cb-57c4-4359-8cdd-6ad784c61428 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:21.369545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9d1cb-57c4-4359-8cdd-6ad784c61428 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:21.369880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9d1cb-57c4-4359-8cdd-6ad784c61428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:21.371909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f2c1dc068f24e4096090a140a1b9561', 'request_id': 'req-1ab763ca-423c-457c-b50d-3dd674732a8a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.395983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-f1a9d1cb-57c4-4359-8cdd-6ad784c61428 None None] MFA Rules not processed for user `bee9fe3603134ce0ab3fbeb8ca4877ea`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:21.397131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f1a9d1cb-57c4-4359-8cdd-6ad784c61428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:21.465232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:21.472927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1371/2755] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:21.492273 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9d1cb-57c4-4359-8cdd-6ad784c61428 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:21.494716 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f1a9d1cb-57c4-4359-8cdd-6ad784c61428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:21.506952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f2c1dc068f24e4096090a140a1b9561', 'request_id': 'req-1ab763ca-423c-457c-b50d-3dd674732a8a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.532312 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] The token cache is being invalidate because project 0f2c1dc068f24e4096090a140a1b9561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:21.533150 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.533933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.536058 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1385/2756] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:21 2022] DELETE /identity/v3/projects/0f2c1dc068f24e4096090a140a1b9561 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:21.540041 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:21.541511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:21.580316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:21.592961 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9d1cb-57c4-4359-8cdd-6ad784c61428 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:21.597141 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928d142-34bf-4154-b0bd-7b6d2743caac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:21.597966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928d142-34bf-4154-b0bd-7b6d2743caac service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:21.598252 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928d142-34bf-4154-b0bd-7b6d2743caac service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:21.598705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928d142-34bf-4154-b0bd-7b6d2743caac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:21.600522 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-d928d142-34bf-4154-b0bd-7b6d2743caac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:21.612705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928d142-34bf-4154-b0bd-7b6d2743caac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:21.614232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d928d142-34bf-4154-b0bd-7b6d2743caac service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:21.623703 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1372/2757] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:04:21.624751 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab763ca-423c-457c-b50d-3dd674732a8a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:21.630107 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:21.631154 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:21.631435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:21.631710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] PATH_INFO: `/v3/users/78e84e5318c3485a966900ff95b782b2` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:21.632957 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e84e5318c3485a966900ff95b782b2)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:21.636555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:21.874488 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e84e5318c3485a966900ff95b782b2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.957145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e84e5318c3485a966900ff95b782b2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.957806 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e84e5318c3485a966900ff95b782b2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.965989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e84e5318c3485a966900ff95b782b2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.966676 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e84e5318c3485a966900ff95b782b2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:21.986647 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1386/2758] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:21 2022] DELETE /identity/v3/users/78e84e5318c3485a966900ff95b782b2 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:22.638683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928d142-34bf-4154-b0bd-7b6d2743caac service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:22.653134 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928d142-34bf-4154-b0bd-7b6d2743caac service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:22.727265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:22.728147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:22.728433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:22.728713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] PATH_INFO: `/v3/projects/4bc178f725dd4aabb2f0f0033f1dc62f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:22.730010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc178f725dd4aabb2f0f0033f1dc62f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:22.731910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:22.888897 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc178f725dd4aabb2f0f0033f1dc62f', 'request_id': 'req-007f9e8c-5543-4367-a8ce-6bfef74e993a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:22.900942 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc178f725dd4aabb2f0f0033f1dc62f', 'request_id': 'req-007f9e8c-5543-4367-a8ce-6bfef74e993a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:23.008367 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc178f725dd4aabb2f0f0033f1dc62f', 'request_id': 'req-007f9e8c-5543-4367-a8ce-6bfef74e993a'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:23.020507 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] The token cache is being invalidate because project 4bc178f725dd4aabb2f0f0033f1dc62f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:23.020796 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:23.021451 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:23.023049 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1373/2759] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:22 2022] DELETE /identity/v3/projects/4bc178f725dd4aabb2f0f0033f1dc62f => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:23.027194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:23.028677 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:23.044309 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f940f9-5cf2-40a9-88be-86d017f6f08f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:23.109244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:23.110899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:23.110899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:23.111103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] PATH_INFO: `/v3/users/347359a24943454782d7e58a3c0492c4` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:23.112218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=347359a24943454782d7e58a3c0492c4)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:23.114176 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:23.346453 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347359a24943454782d7e58a3c0492c4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:23.465680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347359a24943454782d7e58a3c0492c4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:23.466362 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347359a24943454782d7e58a3c0492c4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:23.474512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347359a24943454782d7e58a3c0492c4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:23.475419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ec35f568-e30a-4f82-bb50-32a271267ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347359a24943454782d7e58a3c0492c4'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:23.493644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1387/2760] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:23 2022] DELETE /identity/v3/users/347359a24943454782d7e58a3c0492c4 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:23.575882 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:24.004804 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:24.006675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:24.022853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f9e8c-5543-4367-a8ce-6bfef74e993a admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:24.059155 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:24.087517 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:24.088429 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:24.088715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:24.088993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] PATH_INFO: `/v3/projects/db845f0a44f24d43a205fa8777e0fe9b` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:24.090219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db845f0a44f24d43a205fa8777e0fe9b)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:24.092275 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:24.249484 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db845f0a44f24d43a205fa8777e0fe9b', 'request_id': 'req-8917d0db-08ef-4a03-940d-499f2dedcf33'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:24.254953 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db845f0a44f24d43a205fa8777e0fe9b', 'request_id': 'req-8917d0db-08ef-4a03-940d-499f2dedcf33'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:24.357148 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db845f0a44f24d43a205fa8777e0fe9b', 'request_id': 'req-8917d0db-08ef-4a03-940d-499f2dedcf33'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:24.369882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] The token cache is being invalidate because project db845f0a44f24d43a205fa8777e0fe9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:24.370175 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:24.370815 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-8917d0db-08ef-4a03-940d-499f2dedcf33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:24.372412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1374/2761] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:24 2022] DELETE /identity/v3/projects/db845f0a44f24d43a205fa8777e0fe9b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:24.377971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739dcf1-dcc5-4c7e-bf95-5c60584a2bdf None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:24.378448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739dcf1-dcc5-4c7e-bf95-5c60584a2bdf None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:24.378617 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739dcf1-dcc5-4c7e-bf95-5c60584a2bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:24.397524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-5739dcf1-dcc5-4c7e-bf95-5c60584a2bdf None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:24.398844 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-5739dcf1-dcc5-4c7e-bf95-5c60584a2bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:24.458541 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1388/2762] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:24.468318 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a5e77-06c1-4004-bec2-a2d080c29b55 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:24.468635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a5e77-06c1-4004-bec2-a2d080c29b55 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:24.468927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a5e77-06c1-4004-bec2-a2d080c29b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:24.488445 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-925a5e77-06c1-4004-bec2-a2d080c29b55 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:24.489674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-925a5e77-06c1-4004-bec2-a2d080c29b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:24.497899 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1375/2763] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:24.502975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739dcf1-dcc5-4c7e-bf95-5c60584a2bdf None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:24.512736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739dcf1-dcc5-4c7e-bf95-5c60584a2bdf None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:24.518137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10653a-9aaa-4c2a-9b99-352185d0642a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:24.518794 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10653a-9aaa-4c2a-9b99-352185d0642a admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:24.519078 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10653a-9aaa-4c2a-9b99-352185d0642a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:24.519356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10653a-9aaa-4c2a-9b99-352185d0642a admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:24.520082 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10653a-9aaa-4c2a-9b99-352185d0642a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:24.520391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10653a-9aaa-4c2a-9b99-352185d0642a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:24.522046 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10653a-9aaa-4c2a-9b99-352185d0642a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:24.533793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1389/2764] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:04:24.538228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a5e77-06c1-4004-bec2-a2d080c29b55 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:24.547936 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a5e77-06c1-4004-bec2-a2d080c29b55 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:24.556245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6497c-125f-40d0-aef8-d56c7215e36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:24.556245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6497c-125f-40d0-aef8-d56c7215e36b admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:24.556245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6497c-125f-40d0-aef8-d56c7215e36b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:24.556245 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6497c-125f-40d0-aef8-d56c7215e36b admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:24.556642 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6497c-125f-40d0-aef8-d56c7215e36b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:24.556747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6497c-125f-40d0-aef8-d56c7215e36b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:24.683840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1376/2765] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:24 2022] POST /identity/v3/projects => generated 391 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:24.687358 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10653a-9aaa-4c2a-9b99-352185d0642a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:24.697302 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10653a-9aaa-4c2a-9b99-352185d0642a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:24.753802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32215f1-8f7b-43c6-8d05-f405b90c9c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:24.754433 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32215f1-8f7b-43c6-8d05-f405b90c9c28 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:24.754726 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32215f1-8f7b-43c6-8d05-f405b90c9c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:24.755000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32215f1-8f7b-43c6-8d05-f405b90c9c28 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:24.755685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32215f1-8f7b-43c6-8d05-f405b90c9c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:24.757594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32215f1-8f7b-43c6-8d05-f405b90c9c28 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:24.870533 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1390/2766] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:24 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:24.871696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6497c-125f-40d0-aef8-d56c7215e36b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:24.884265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6497c-125f-40d0-aef8-d56c7215e36b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:24.889177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1954c-5567-4487-8478-acd2831caca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:24.889941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1954c-5567-4487-8478-acd2831caca5 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:24.890228 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1954c-5567-4487-8478-acd2831caca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:24.890503 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1954c-5567-4487-8478-acd2831caca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:24.891281 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1954c-5567-4487-8478-acd2831caca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:24.892982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1954c-5567-4487-8478-acd2831caca5 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:24.903788 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1377/2767] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:24.908099 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32215f1-8f7b-43c6-8d05-f405b90c9c28 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:24.918323 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32215f1-8f7b-43c6-8d05-f405b90c9c28 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:24.959085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b015d-4472-4a84-9d16-41c3de7a56fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:24.959085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b015d-4472-4a84-9d16-41c3de7a56fe admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:24.959085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b015d-4472-4a84-9d16-41c3de7a56fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:24.959085 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b015d-4472-4a84-9d16-41c3de7a56fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:24.959519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b015d-4472-4a84-9d16-41c3de7a56fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:24.959519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b015d-4472-4a84-9d16-41c3de7a56fe admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:24.959519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1391/2768] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:24.962218 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1954c-5567-4487-8478-acd2831caca5 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:24.972846 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1954c-5567-4487-8478-acd2831caca5 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:24.977696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2db8c1-598e-4c42-abde-8f14484e651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:24.978731 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2db8c1-598e-4c42-abde-8f14484e651d admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:24.979012 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2db8c1-598e-4c42-abde-8f14484e651d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:24.979292 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2db8c1-598e-4c42-abde-8f14484e651d admin admin] PATH_INFO: `/v3/projects/29ae24dcdaed4c73911a0ee797e9c496/users/a4498ae758ec4eb3ae945911999fbd9b/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:24.995990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2db8c1-598e-4c42-abde-8f14484e651d admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ae24dcdaed4c73911a0ee797e9c496, user_id=a4498ae758ec4eb3ae945911999fbd9b, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:24.999433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2db8c1-598e-4c42-abde-8f14484e651d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:25.129646 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1378/2769] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:24 2022] PUT /identity/v3/projects/29ae24dcdaed4c73911a0ee797e9c496/users/a4498ae758ec4eb3ae945911999fbd9b/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:25.140139 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278293f7-b252-463c-bbf3-ee784b8217c7 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:25.140435 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278293f7-b252-463c-bbf3-ee784b8217c7 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:25.140712 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278293f7-b252-463c-bbf3-ee784b8217c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:25.168551 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-278293f7-b252-463c-bbf3-ee784b8217c7 None None] MFA Rules not processed for user `a4498ae758ec4eb3ae945911999fbd9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:25.169779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-278293f7-b252-463c-bbf3-ee784b8217c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:25.238489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1392/2770] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:25.263419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2db8c1-598e-4c42-abde-8f14484e651d admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:25.265335 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-8d2db8c1-598e-4c42-abde-8f14484e651d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:25.328177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2db8c1-598e-4c42-abde-8f14484e651d admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:25.333046 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6a174-2d20-4042-807b-a3808d59a0e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:25.333884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6a174-2d20-4042-807b-a3808d59a0e2 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:25.334353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6a174-2d20-4042-807b-a3808d59a0e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:25.334685 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6a174-2d20-4042-807b-a3808d59a0e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:25.345226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6a174-2d20-4042-807b-a3808d59a0e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:25.346839 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6a174-2d20-4042-807b-a3808d59a0e2 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:25.356586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1379/2771] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:25.763169 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:25.904250 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:33.826715 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278293f7-b252-463c-bbf3-ee784b8217c7 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:33.828370 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-278293f7-b252-463c-bbf3-ee784b8217c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:33.894719 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278293f7-b252-463c-bbf3-ee784b8217c7 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:33.899441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:33.900378 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:33.900666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:33.900945 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] PATH_INFO: `/v3/users/bee9fe3603134ce0ab3fbeb8ca4877ea` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:33.902218 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bee9fe3603134ce0ab3fbeb8ca4877ea)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:33.904259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:34.136297 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bee9fe3603134ce0ab3fbeb8ca4877ea'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:34.225355 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bee9fe3603134ce0ab3fbeb8ca4877ea'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:34.225998 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bee9fe3603134ce0ab3fbeb8ca4877ea'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:34.227432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:34.233944 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bee9fe3603134ce0ab3fbeb8ca4877ea'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:34.235103 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bee9fe3603134ce0ab3fbeb8ca4877ea'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:34.252277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1393/2772] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:33 2022] DELETE /identity/v3/users/bee9fe3603134ce0ab3fbeb8ca4877ea => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:34.257003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc9800-180e-4c7a-9d5e-5fe037f6cc8f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:34.257515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc9800-180e-4c7a-9d5e-5fe037f6cc8f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:34.257626 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc9800-180e-4c7a-9d5e-5fe037f6cc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:34.274943 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-99bc9800-180e-4c7a-9d5e-5fe037f6cc8f None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:34.276162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-99bc9800-180e-4c7a-9d5e-5fe037f6cc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:34.338520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1380/2773] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:34.349097 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:34.350574 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:34.410877 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48515750-fce4-4dcd-aed4-8887a9637f65 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:34.414373 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7454a-0988-440a-bd54-c273e21ba035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:34.415036 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7454a-0988-440a-bd54-c273e21ba035 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:34.415338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7454a-0988-440a-bd54-c273e21ba035 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:34.415671 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7454a-0988-440a-bd54-c273e21ba035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:34.425252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7454a-0988-440a-bd54-c273e21ba035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:34.426729 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7454a-0988-440a-bd54-c273e21ba035 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:34.438396 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1394/2774] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:34.989869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc9800-180e-4c7a-9d5e-5fe037f6cc8f None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:34.989869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-99bc9800-180e-4c7a-9d5e-5fe037f6cc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:34.996776 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc9800-180e-4c7a-9d5e-5fe037f6cc8f None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:35.005291 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:35.006349 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:35.006647 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:35.006927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] PATH_INFO: `/v3/projects/31d5c1320c1646769d9b4d7322c0a6ee` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:35.008205 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d5c1320c1646769d9b4d7322c0a6ee)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:35.010270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:35.177149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d5c1320c1646769d9b4d7322c0a6ee', 'request_id': 'req-9217d019-7f48-42d5-88ee-e8147f794603'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:35.183854 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d5c1320c1646769d9b4d7322c0a6ee', 'request_id': 'req-9217d019-7f48-42d5-88ee-e8147f794603'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:35.259952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d5c1320c1646769d9b4d7322c0a6ee', 'request_id': 'req-9217d019-7f48-42d5-88ee-e8147f794603'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:35.272202 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] The token cache is being invalidate because project 31d5c1320c1646769d9b4d7322c0a6ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:35.272486 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:35.273119 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9217d019-7f48-42d5-88ee-e8147f794603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:35.274738 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1381/2775] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:34 2022] DELETE /identity/v3/projects/31d5c1320c1646769d9b4d7322c0a6ee => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:35.279797 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee7dac-da62-4d8c-a51b-7cb1d75da95b None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:35.280243 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee7dac-da62-4d8c-a51b-7cb1d75da95b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:35.280505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee7dac-da62-4d8c-a51b-7cb1d75da95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:35.299263 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-18ee7dac-da62-4d8c-a51b-7cb1d75da95b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:35.300502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-18ee7dac-da62-4d8c-a51b-7cb1d75da95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:35.362043 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1395/2776] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:35.369956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3279a-4fb5-4827-b57b-e5b79eeffe5b None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:35.370251 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3279a-4fb5-4827-b57b-e5b79eeffe5b None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:35.370534 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3279a-4fb5-4827-b57b-e5b79eeffe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:35.389630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-05f3279a-4fb5-4827-b57b-e5b79eeffe5b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:35.390889 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-05f3279a-4fb5-4827-b57b-e5b79eeffe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:35.399108 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1382/2777] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:35.403372 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee7dac-da62-4d8c-a51b-7cb1d75da95b None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:35.413329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee7dac-da62-4d8c-a51b-7cb1d75da95b None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:35.417966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32b246-132f-47e3-a05b-cb1c1c2f6bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:35.418630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32b246-132f-47e3-a05b-cb1c1c2f6bcf admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:35.418917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32b246-132f-47e3-a05b-cb1c1c2f6bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:35.419197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32b246-132f-47e3-a05b-cb1c1c2f6bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:35.419952 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32b246-132f-47e3-a05b-cb1c1c2f6bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:35.420261 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32b246-132f-47e3-a05b-cb1c1c2f6bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:35.422789 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32b246-132f-47e3-a05b-cb1c1c2f6bcf admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:35.435299 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1396/2778] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:04:35.440158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3279a-4fb5-4827-b57b-e5b79eeffe5b None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:35.449403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3279a-4fb5-4827-b57b-e5b79eeffe5b None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:35.454157 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4e055-64c3-48ff-b4f7-c539dff42ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:35.454766 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4e055-64c3-48ff-b4f7-c539dff42ded admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:35.455048 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4e055-64c3-48ff-b4f7-c539dff42ded admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:35.455329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4e055-64c3-48ff-b4f7-c539dff42ded admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:35.456015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4e055-64c3-48ff-b4f7-c539dff42ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:35.457935 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4e055-64c3-48ff-b4f7-c539dff42ded admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:35.543109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1383/2779] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:35 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:35.545547 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32b246-132f-47e3-a05b-cb1c1c2f6bcf admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:35.556779 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32b246-132f-47e3-a05b-cb1c1c2f6bcf admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:35.618823 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61627cb-a817-401e-8e57-484ccecdd21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:35.619505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61627cb-a817-401e-8e57-484ccecdd21a admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:35.619817 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61627cb-a817-401e-8e57-484ccecdd21a admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:35.620100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61627cb-a817-401e-8e57-484ccecdd21a admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:35.620808 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61627cb-a817-401e-8e57-484ccecdd21a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:35.622771 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61627cb-a817-401e-8e57-484ccecdd21a admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:35.630426 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:35.751344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1397/2780] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:35 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:35.756018 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4e055-64c3-48ff-b4f7-c539dff42ded admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:35.767011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4e055-64c3-48ff-b4f7-c539dff42ded admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:35.771929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eca873-e2c3-4a1d-b701-e2c72164d6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:35.772601 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eca873-e2c3-4a1d-b701-e2c72164d6be admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:35.772887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eca873-e2c3-4a1d-b701-e2c72164d6be admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:35.773178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eca873-e2c3-4a1d-b701-e2c72164d6be admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:35.773938 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eca873-e2c3-4a1d-b701-e2c72164d6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:35.778723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eca873-e2c3-4a1d-b701-e2c72164d6be admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:35.786803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1384/2781] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:35.791270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61627cb-a817-401e-8e57-484ccecdd21a admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:35.803141 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61627cb-a817-401e-8e57-484ccecdd21a admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:35.807697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b46c7-ed71-4f29-ba37-6c8e93508f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:35.808359 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b46c7-ed71-4f29-ba37-6c8e93508f42 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:35.808644 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b46c7-ed71-4f29-ba37-6c8e93508f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:35.813376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b46c7-ed71-4f29-ba37-6c8e93508f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:35.813376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b46c7-ed71-4f29-ba37-6c8e93508f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:35.813376 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b46c7-ed71-4f29-ba37-6c8e93508f42 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:35.823094 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1398/2782] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:35.827356 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eca873-e2c3-4a1d-b701-e2c72164d6be admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:35.837966 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eca873-e2c3-4a1d-b701-e2c72164d6be admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:35.842927 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647dd29-5a9d-4947-8b17-fca97123bda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:35.843826 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647dd29-5a9d-4947-8b17-fca97123bda5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:35.844271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647dd29-5a9d-4947-8b17-fca97123bda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:35.844604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647dd29-5a9d-4947-8b17-fca97123bda5 admin admin] PATH_INFO: `/v3/projects/50ea7e761b9c4208aad32750cc4c6906/users/ae461bdf03fb4364b707982e592f697a/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:35.861267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647dd29-5a9d-4947-8b17-fca97123bda5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50ea7e761b9c4208aad32750cc4c6906, user_id=ae461bdf03fb4364b707982e592f697a, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:35.864033 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647dd29-5a9d-4947-8b17-fca97123bda5 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:35.993305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1385/2783] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:35 2022] PUT /identity/v3/projects/50ea7e761b9c4208aad32750cc4c6906/users/ae461bdf03fb4364b707982e592f697a/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:36.064710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d631f-0d05-40bf-9afe-e4f66654a1fb None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:36.064940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d631f-0d05-40bf-9afe-e4f66654a1fb None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:36.065156 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d631f-0d05-40bf-9afe-e4f66654a1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:36.092799 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-915d631f-0d05-40bf-9afe-e4f66654a1fb None None] MFA Rules not processed for user `ae461bdf03fb4364b707982e592f697a`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:36.094440 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-915d631f-0d05-40bf-9afe-e4f66654a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:36.161915 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1399/2784] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:36.173664 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647dd29-5a9d-4947-8b17-fca97123bda5 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:36.175539 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b647dd29-5a9d-4947-8b17-fca97123bda5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:36.198802 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:36.234791 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647dd29-5a9d-4947-8b17-fca97123bda5 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:36.239037 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e4885-0d7c-4f9b-a2d6-eec13c567520 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:36.239688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e4885-0d7c-4f9b-a2d6-eec13c567520 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:36.239972 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e4885-0d7c-4f9b-a2d6-eec13c567520 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:36.240249 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e4885-0d7c-4f9b-a2d6-eec13c567520 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:36.249014 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e4885-0d7c-4f9b-a2d6-eec13c567520 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:36.250476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e4885-0d7c-4f9b-a2d6-eec13c567520 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:36.259358 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1386/2785] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:37.887065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d631f-0d05-40bf-9afe-e4f66654a1fb None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:37.888607 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-915d631f-0d05-40bf-9afe-e4f66654a1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:37.961238 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d631f-0d05-40bf-9afe-e4f66654a1fb None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:37.966873 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb7e37-65ec-40bf-9927-ed4da5c29a89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:37.967649 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb7e37-65ec-40bf-9927-ed4da5c29a89 service placement] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:37.968027 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb7e37-65ec-40bf-9927-ed4da5c29a89 service placement] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:37.968259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb7e37-65ec-40bf-9927-ed4da5c29a89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:37.970226 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-35cb7e37-65ec-40bf-9927-ed4da5c29a89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:37.982059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb7e37-65ec-40bf-9927-ed4da5c29a89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:37.983911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb7e37-65ec-40bf-9927-ed4da5c29a89 service placement] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:37.993621 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1400/2786] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:38.747142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e4885-0d7c-4f9b-a2d6-eec13c567520 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:38.752705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-534e4885-0d7c-4f9b-a2d6-eec13c567520 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:38.820198 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e4885-0d7c-4f9b-a2d6-eec13c567520 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:38.824805 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:38.825723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:38.826117 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:38.826393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] PATH_INFO: `/v3/users/b6e8372460404eef934cd8b6bc7a290f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:38.827607 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e8372460404eef934cd8b6bc7a290f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:38.830271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:39.211747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e8372460404eef934cd8b6bc7a290f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.293529 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e8372460404eef934cd8b6bc7a290f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.297715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e8372460404eef934cd8b6bc7a290f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.302786 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e8372460404eef934cd8b6bc7a290f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.303452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e8372460404eef934cd8b6bc7a290f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.323107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1387/2787] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:38 2022] DELETE /identity/v3/users/b6e8372460404eef934cd8b6bc7a290f => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:39.621623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb7e37-65ec-40bf-9927-ed4da5c29a89 service placement] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:39.636104 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb7e37-65ec-40bf-9927-ed4da5c29a89 service placement] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:39.698178 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:39.699061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:39.699345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:39.699624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] PATH_INFO: `/v3/projects/1a14d18943b64d8a914952bd1c0e05e1` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:39.700842 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a14d18943b64d8a914952bd1c0e05e1)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:39.703026 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:39.706526 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:39.803658 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a14d18943b64d8a914952bd1c0e05e1', 'request_id': 'req-01ff6049-51eb-4be5-8da5-116c8f92fd7f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.809908 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a14d18943b64d8a914952bd1c0e05e1', 'request_id': 'req-01ff6049-51eb-4be5-8da5-116c8f92fd7f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.916583 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a14d18943b64d8a914952bd1c0e05e1', 'request_id': 'req-01ff6049-51eb-4be5-8da5-116c8f92fd7f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.929467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] The token cache is being invalidate because project 1a14d18943b64d8a914952bd1c0e05e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:39.929760 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.930425 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:39.931980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1401/2788] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:39 2022] DELETE /identity/v3/projects/1a14d18943b64d8a914952bd1c0e05e1 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:39.936019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:39.937412 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:39.951181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2442d3-87f6-465d-9ebf-4677eb1c8a23 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:39.993666 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:40.010125 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:40.011028 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:40.011311 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:40.011586 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] PATH_INFO: `/v3/users/002642649f854d2a8fab486e3d7f8883` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:40.012807 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=002642649f854d2a8fab486e3d7f8883)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:40.014801 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:40.281793 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002642649f854d2a8fab486e3d7f8883'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:40.402433 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002642649f854d2a8fab486e3d7f8883'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:40.403061 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002642649f854d2a8fab486e3d7f8883'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:40.411446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002642649f854d2a8fab486e3d7f8883'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:40.412489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-98b0e1f0-c6dc-4b2c-a647-abb6dd382e8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002642649f854d2a8fab486e3d7f8883'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:40.430629 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1388/2789] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:39 2022] DELETE /identity/v3/users/002642649f854d2a8fab486e3d7f8883 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:40.696500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:40.703155 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:40.712921 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff6049-51eb-4be5-8da5-116c8f92fd7f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:40.779890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:40.780865 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:40.781110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:40.781395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] PATH_INFO: `/v3/projects/1021cb5b8d5843bc85cf553692a7b2e8` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:40.782619 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1021cb5b8d5843bc85cf553692a7b2e8)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:40.784519 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:40.969489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1021cb5b8d5843bc85cf553692a7b2e8', 'request_id': 'req-27cba73c-5012-4e24-b2ef-18c39b45c23f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:40.975109 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1021cb5b8d5843bc85cf553692a7b2e8', 'request_id': 'req-27cba73c-5012-4e24-b2ef-18c39b45c23f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:41.105767 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1021cb5b8d5843bc85cf553692a7b2e8', 'request_id': 'req-27cba73c-5012-4e24-b2ef-18c39b45c23f'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:41.118254 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] The token cache is being invalidate because project 1021cb5b8d5843bc85cf553692a7b2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:41.118544 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:41.119194 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-27cba73c-5012-4e24-b2ef-18c39b45c23f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:41.120745 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1402/2790] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:40 2022] DELETE /identity/v3/projects/1021cb5b8d5843bc85cf553692a7b2e8 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:41.127166 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c156ae-b337-4b79-a103-7f753a54a151 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.127650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c156ae-b337-4b79-a103-7f753a54a151 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.127771 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c156ae-b337-4b79-a103-7f753a54a151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.146229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-55c156ae-b337-4b79-a103-7f753a54a151 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:41.147473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-55c156ae-b337-4b79-a103-7f753a54a151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:41.211370 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1389/2791] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:41.219345 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eefd28-5ff6-4919-b6ff-0fae16df033b None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.219683 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eefd28-5ff6-4919-b6ff-0fae16df033b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.219911 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eefd28-5ff6-4919-b6ff-0fae16df033b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.239356 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-13eefd28-5ff6-4919-b6ff-0fae16df033b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:41.240709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-13eefd28-5ff6-4919-b6ff-0fae16df033b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:41.248774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1403/2792] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:41.253941 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c156ae-b337-4b79-a103-7f753a54a151 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:41.265520 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c156ae-b337-4b79-a103-7f753a54a151 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:41.270589 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0ea14-d66a-4689-9b56-33e0470c2352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:41.271313 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0ea14-d66a-4689-9b56-33e0470c2352 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.271604 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0ea14-d66a-4689-9b56-33e0470c2352 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.271884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0ea14-d66a-4689-9b56-33e0470c2352 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.272630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0ea14-d66a-4689-9b56-33e0470c2352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:41.272939 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0ea14-d66a-4689-9b56-33e0470c2352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:41.274641 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0ea14-d66a-4689-9b56-33e0470c2352 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:41.286992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1390/2793] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:04:41.291512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eefd28-5ff6-4919-b6ff-0fae16df033b None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:41.302168 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eefd28-5ff6-4919-b6ff-0fae16df033b None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:41.307248 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b4001-3510-4163-8fbb-72d490a70486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:41.307909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4001-3510-4163-8fbb-72d490a70486 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.308197 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4001-3510-4163-8fbb-72d490a70486 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.308482 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b4001-3510-4163-8fbb-72d490a70486 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.309210 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b4001-3510-4163-8fbb-72d490a70486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:41.311199 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b4001-3510-4163-8fbb-72d490a70486 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:41.325584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:41.419962 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1404/2794] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:41 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:41.425010 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0ea14-d66a-4689-9b56-33e0470c2352 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:41.435187 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0ea14-d66a-4689-9b56-33e0470c2352 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:41.481753 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:41.494180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364cd11-2539-4402-8b2f-a6e48e30f4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:41.494859 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364cd11-2539-4402-8b2f-a6e48e30f4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.495144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364cd11-2539-4402-8b2f-a6e48e30f4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.495419 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364cd11-2539-4402-8b2f-a6e48e30f4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.496109 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364cd11-2539-4402-8b2f-a6e48e30f4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:41.498258 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364cd11-2539-4402-8b2f-a6e48e30f4e2 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:41.615213 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1391/2795] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:41 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:41.619578 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b4001-3510-4163-8fbb-72d490a70486 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:41.629965 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b4001-3510-4163-8fbb-72d490a70486 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:41.637032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949241fc-e2f8-4ff0-ab41-00e9c165f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:41.637032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949241fc-e2f8-4ff0-ab41-00e9c165f358 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.637032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949241fc-e2f8-4ff0-ab41-00e9c165f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.637032 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949241fc-e2f8-4ff0-ab41-00e9c165f358 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.637479 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949241fc-e2f8-4ff0-ab41-00e9c165f358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:41.638553 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949241fc-e2f8-4ff0-ab41-00e9c165f358 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:41.650369 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1405/2796] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:41.654590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364cd11-2539-4402-8b2f-a6e48e30f4e2 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:41.665395 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364cd11-2539-4402-8b2f-a6e48e30f4e2 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:41.670378 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c986b1-b476-4708-97da-d1361e03679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:41.671346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c986b1-b476-4708-97da-d1361e03679e admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.671715 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c986b1-b476-4708-97da-d1361e03679e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.672650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c986b1-b476-4708-97da-d1361e03679e admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.673449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c986b1-b476-4708-97da-d1361e03679e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:41.675050 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c986b1-b476-4708-97da-d1361e03679e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:41.686121 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1392/2797] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:41.690081 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949241fc-e2f8-4ff0-ab41-00e9c165f358 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:41.700177 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949241fc-e2f8-4ff0-ab41-00e9c165f358 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:41.707343 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f718cb-1ebf-404b-bfc3-a6b949b6fd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:41.708254 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f718cb-1ebf-404b-bfc3-a6b949b6fd45 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.708536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f718cb-1ebf-404b-bfc3-a6b949b6fd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.708813 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f718cb-1ebf-404b-bfc3-a6b949b6fd45 admin admin] PATH_INFO: `/v3/projects/2d2680a4e4cd4ee3b10b0252fb146518/users/1cfee2de424f41cab7daeecb7e60a4c8/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.725127 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f718cb-1ebf-404b-bfc3-a6b949b6fd45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2680a4e4cd4ee3b10b0252fb146518, user_id=1cfee2de424f41cab7daeecb7e60a4c8, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:41.727898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f718cb-1ebf-404b-bfc3-a6b949b6fd45 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:41.797901 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1406/2798] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:41 2022] PUT /identity/v3/projects/2d2680a4e4cd4ee3b10b0252fb146518/users/1cfee2de424f41cab7daeecb7e60a4c8/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:41.809194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e030a2-71fe-4b98-ba9a-e7fcb739bea7 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.809331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e030a2-71fe-4b98-ba9a-e7fcb739bea7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.809644 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e030a2-71fe-4b98-ba9a-e7fcb739bea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.832235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f718cb-1ebf-404b-bfc3-a6b949b6fd45 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:41.838492 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e0e030a2-71fe-4b98-ba9a-e7fcb739bea7 None None] MFA Rules not processed for user `1cfee2de424f41cab7daeecb7e60a4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:41.839710 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e0e030a2-71fe-4b98-ba9a-e7fcb739bea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:41.841592 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-57f718cb-1ebf-404b-bfc3-a6b949b6fd45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:41.924341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f718cb-1ebf-404b-bfc3-a6b949b6fd45 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:41.929111 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1393/2799] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:41.931700 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:41.933076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.933660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.933887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e030a2-71fe-4b98-ba9a-e7fcb739bea7 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:41.934266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] PATH_INFO: `/v3/users/ae461bdf03fb4364b707982e592f697a` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.935743 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae461bdf03fb4364b707982e592f697a)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:41.943783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:41.950015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e030a2-71fe-4b98-ba9a-e7fcb739bea7 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:41.955149 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd893299-15c3-48b4-8ca8-8d19d273fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:41.960842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd893299-15c3-48b4-8ca8-8d19d273fa6e admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:41.960842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd893299-15c3-48b4-8ca8-8d19d273fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:41.960842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd893299-15c3-48b4-8ca8-8d19d273fa6e admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:41.960842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd893299-15c3-48b4-8ca8-8d19d273fa6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:41.960842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd893299-15c3-48b4-8ca8-8d19d273fa6e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:42.141498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1394/2800] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:41 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:42.145512 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd893299-15c3-48b4-8ca8-8d19d273fa6e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:42.191163 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae461bdf03fb4364b707982e592f697a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:42.217848 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd893299-15c3-48b4-8ca8-8d19d273fa6e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:42.222326 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2067ef-a818-4233-bb1a-4cb9014b8787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:42.223011 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2067ef-a818-4233-bb1a-4cb9014b8787 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:42.223296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2067ef-a818-4233-bb1a-4cb9014b8787 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:42.223599 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2067ef-a818-4233-bb1a-4cb9014b8787 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:42.224400 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2067ef-a818-4233-bb1a-4cb9014b8787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:42.226216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2067ef-a818-4233-bb1a-4cb9014b8787 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:42.276750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae461bdf03fb4364b707982e592f697a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:42.277537 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae461bdf03fb4364b707982e592f697a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:42.285943 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae461bdf03fb4364b707982e592f697a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:42.286704 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-ee965295-4275-4fe1-9f36-993ebb8a51b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae461bdf03fb4364b707982e592f697a'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:42.309038 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1407/2801] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:41 2022] DELETE /identity/v3/users/ae461bdf03fb4364b707982e592f697a => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:42.314291 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00d1f4-3898-45c1-bb6c-ab3d12605b45 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:42.314513 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00d1f4-3898-45c1-bb6c-ab3d12605b45 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:42.314794 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00d1f4-3898-45c1-bb6c-ab3d12605b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:42.329297 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1395/2802] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:42 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:42.331377 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2067ef-a818-4233-bb1a-4cb9014b8787 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:42.333239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-6c2067ef-a818-4233-bb1a-4cb9014b8787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:42.341329 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ed00d1f4-3898-45c1-bb6c-ab3d12605b45 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:42.342674 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ed00d1f4-3898-45c1-bb6c-ab3d12605b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:42.347242 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2067ef-a818-4233-bb1a-4cb9014b8787 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:42.371757 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:42.391648 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:42.419688 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d605a24-e665-49d3-81c5-6df81778c091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:42.420469 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d605a24-e665-49d3-81c5-6df81778c091 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:42.420830 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d605a24-e665-49d3-81c5-6df81778c091 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:42.421498 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d605a24-e665-49d3-81c5-6df81778c091 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:42.422343 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d605a24-e665-49d3-81c5-6df81778c091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:42.424229 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d605a24-e665-49d3-81c5-6df81778c091 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:42.430900 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1408/2803] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:42.442408 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00d1f4-3898-45c1-bb6c-ab3d12605b45 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:42.447510 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ed00d1f4-3898-45c1-bb6c-ab3d12605b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:42.447623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1396/2804] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:42.450387 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d605a24-e665-49d3-81c5-6df81778c091 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:42.462216 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d605a24-e665-49d3-81c5-6df81778c091 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:42.467692 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03789-3edf-4080-ab96-140bebb32b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:42.468770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03789-3edf-4080-ab96-140bebb32b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:42.469158 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03789-3edf-4080-ab96-140bebb32b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:42.469541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c03789-3edf-4080-ab96-140bebb32b37 admin admin] PATH_INFO: `/v3/projects/9ed30f8779ac4acfbfa9f060a5826645/users/f0aee9238b764b2e87d65a837e347b36/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:42.486437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c03789-3edf-4080-ab96-140bebb32b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed30f8779ac4acfbfa9f060a5826645, user_id=f0aee9238b764b2e87d65a837e347b36, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:42.490290 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c03789-3edf-4080-ab96-140bebb32b37 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:42.503570 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00d1f4-3898-45c1-bb6c-ab3d12605b45 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:42.507179 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6eff2f-6f4b-4697-be6d-681c4992dcf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:42.507840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eff2f-6f4b-4697-be6d-681c4992dcf9 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:42.508122 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eff2f-6f4b-4697-be6d-681c4992dcf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:42.508401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6eff2f-6f4b-4697-be6d-681c4992dcf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:42.518148 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6eff2f-6f4b-4697-be6d-681c4992dcf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:42.519584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6eff2f-6f4b-4697-be6d-681c4992dcf9 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:42.529237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1409/2805] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:04:42.687872 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1397/2806] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:42 2022] PUT /identity/v3/projects/9ed30f8779ac4acfbfa9f060a5826645/users/f0aee9238b764b2e87d65a837e347b36/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:42.691891 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6eff2f-6f4b-4697-be6d-681c4992dcf9 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:42.701630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6eff2f-6f4b-4697-be6d-681c4992dcf9 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:42.763733 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f6f90-a284-4c7c-9d63-bc11267e7a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:42.764971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f6f90-a284-4c7c-9d63-bc11267e7a9f admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:42.765980 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f6f90-a284-4c7c-9d63-bc11267e7a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:42.766164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f6f90-a284-4c7c-9d63-bc11267e7a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:42.766838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f6f90-a284-4c7c-9d63-bc11267e7a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:42.768567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f6f90-a284-4c7c-9d63-bc11267e7a9f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:42.779887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1410/2807] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:42.785642 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03789-3edf-4080-ab96-140bebb32b37 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:42.799450 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c03789-3edf-4080-ab96-140bebb32b37 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:42.804696 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a3b56-4be8-4e7f-ad11-2d33786b1598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:42.805955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a3b56-4be8-4e7f-ad11-2d33786b1598 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:42.806353 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a3b56-4be8-4e7f-ad11-2d33786b1598 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:42.806765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a3b56-4be8-4e7f-ad11-2d33786b1598 admin admin] PATH_INFO: `/v3/projects/9ed30f8779ac4acfbfa9f060a5826645/users/f0aee9238b764b2e87d65a837e347b36/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:42.808891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a3b56-4be8-4e7f-ad11-2d33786b1598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed30f8779ac4acfbfa9f060a5826645, user_id=f0aee9238b764b2e87d65a837e347b36, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:42.811636 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a3b56-4be8-4e7f-ad11-2d33786b1598 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:42.824270 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1398/2808] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:42 2022] PUT /identity/v3/projects/9ed30f8779ac4acfbfa9f060a5826645/users/f0aee9238b764b2e87d65a837e347b36/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:42.835568 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de5d5-f6d0-47eb-831f-8e8816c458a6 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:42.836055 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de5d5-f6d0-47eb-831f-8e8816c458a6 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:42.836286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197de5d5-f6d0-47eb-831f-8e8816c458a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:42.866365 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-197de5d5-f6d0-47eb-831f-8e8816c458a6 None None] MFA Rules not processed for user `f0aee9238b764b2e87d65a837e347b36`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:42.867616 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-197de5d5-f6d0-47eb-831f-8e8816c458a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:42.941419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1411/2809] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:42.952284 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a3b56-4be8-4e7f-ad11-2d33786b1598 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:42.954042 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-bf9a3b56-4be8-4e7f-ad11-2d33786b1598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:43.010110 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=100 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:43.015646 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de5d5-f6d0-47eb-831f-8e8816c458a6 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:43.017387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-197de5d5-f6d0-47eb-831f-8e8816c458a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:43.019024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a3b56-4be8-4e7f-ad11-2d33786b1598 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:43.024170 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fef74-6c4c-41ed-9dee-2d5100f3c258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:43.025019 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fef74-6c4c-41ed-9dee-2d5100f3c258 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:43.025376 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fef74-6c4c-41ed-9dee-2d5100f3c258 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:43.025853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fef74-6c4c-41ed-9dee-2d5100f3c258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:43.028259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-007fef74-6c4c-41ed-9dee-2d5100f3c258 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:43.095119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197de5d5-f6d0-47eb-831f-8e8816c458a6 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:43.100493 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:43.102546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:43.102546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:43.102546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] PATH_INFO: `/v3/projects/50ea7e761b9c4208aad32750cc4c6906` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:43.103234 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] RBAC: Authorizing `identity:delete_project(project_id=50ea7e761b9c4208aad32750cc4c6906)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:43.105152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:43.110338 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fef74-6c4c-41ed-9dee-2d5100f3c258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:43.111891 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fef74-6c4c-41ed-9dee-2d5100f3c258 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:43.121671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1399/2810] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:43.325173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50ea7e761b9c4208aad32750cc4c6906', 'request_id': 'req-fdabfc42-a509-428e-bc51-1df2a805516d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:43.334888 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50ea7e761b9c4208aad32750cc4c6906', 'request_id': 'req-fdabfc42-a509-428e-bc51-1df2a805516d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:43.490645 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50ea7e761b9c4208aad32750cc4c6906', 'request_id': 'req-fdabfc42-a509-428e-bc51-1df2a805516d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:43.505174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] The token cache is being invalidate because project 50ea7e761b9c4208aad32750cc4c6906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:43.505462 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:43.506188 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fdabfc42-a509-428e-bc51-1df2a805516d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:43.507777 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1412/2811] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:43 2022] DELETE /identity/v3/projects/50ea7e761b9c4208aad32750cc4c6906 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:46.450186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fef74-6c4c-41ed-9dee-2d5100f3c258 service cinder] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:46.466983 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fef74-6c4c-41ed-9dee-2d5100f3c258 service cinder] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:46.506465 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:46.529910 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:46.531480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:46.531869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:46.532160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] PATH_INFO: `/v3/users/1cfee2de424f41cab7daeecb7e60a4c8` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:46.533381 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cfee2de424f41cab7daeecb7e60a4c8)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:46.535489 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:46.778709 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cfee2de424f41cab7daeecb7e60a4c8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:46.911361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cfee2de424f41cab7daeecb7e60a4c8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:46.912139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cfee2de424f41cab7daeecb7e60a4c8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:46.920675 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cfee2de424f41cab7daeecb7e60a4c8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:46.921594 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cfee2de424f41cab7daeecb7e60a4c8'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:46.940115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1400/2812] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:46 2022] DELETE /identity/v3/users/1cfee2de424f41cab7daeecb7e60a4c8 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:46.944812 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcdedc-8314-462b-bf2b-93fbbb69e0bb None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:46.945331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcdedc-8314-462b-bf2b-93fbbb69e0bb None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:46.945436 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcdedc-8314-462b-bf2b-93fbbb69e0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:46.963594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-b5dcdedc-8314-462b-bf2b-93fbbb69e0bb None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:46.964793 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b5dcdedc-8314-462b-bf2b-93fbbb69e0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:47.027303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1413/2813] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:47.037956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:47.041635 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:47.100773 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b7326-048c-4a08-86e6-81f58eb92a62 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:47.104427 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092329c-ca49-4602-bb04-2c8105cec86d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:47.105075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092329c-ca49-4602-bb04-2c8105cec86d service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:47.105361 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092329c-ca49-4602-bb04-2c8105cec86d service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:47.105642 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092329c-ca49-4602-bb04-2c8105cec86d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:47.114825 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092329c-ca49-4602-bb04-2c8105cec86d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:47.116259 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092329c-ca49-4602-bb04-2c8105cec86d service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:47.127922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1401/2814] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:47.510034 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcdedc-8314-462b-bf2b-93fbbb69e0bb None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:47.511467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-b5dcdedc-8314-462b-bf2b-93fbbb69e0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:47.524229 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcdedc-8314-462b-bf2b-93fbbb69e0bb None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:47.529185 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:47.530100 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:47.530762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:47.530762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] PATH_INFO: `/v3/users/a4498ae758ec4eb3ae945911999fbd9b` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:47.535241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4498ae758ec4eb3ae945911999fbd9b)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:47.535241 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:47.688618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092329c-ca49-4602-bb04-2c8105cec86d service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:47.690835 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e092329c-ca49-4602-bb04-2c8105cec86d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:47.707190 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092329c-ca49-4602-bb04-2c8105cec86d service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:47.723417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:47.723417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:47.723417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:47.723417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] PATH_INFO: `/v3/projects/2d2680a4e4cd4ee3b10b0252fb146518` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:47.724240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d2680a4e4cd4ee3b10b0252fb146518)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:47.724240 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:47.746879 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:47.895196 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4498ae758ec4eb3ae945911999fbd9b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.012664 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4498ae758ec4eb3ae945911999fbd9b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.014202 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4498ae758ec4eb3ae945911999fbd9b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.017182 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d2680a4e4cd4ee3b10b0252fb146518', 'request_id': 'req-e333482b-a240-4a0b-9045-7802ffd4338f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.021374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:48.023054 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4498ae758ec4eb3ae945911999fbd9b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.023722 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-75f1ef1f-445a-4f94-b3c6-e24af90913e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4498ae758ec4eb3ae945911999fbd9b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.024003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d2680a4e4cd4ee3b10b0252fb146518', 'request_id': 'req-e333482b-a240-4a0b-9045-7802ffd4338f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.043747 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1414/2815] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:47 2022] DELETE /identity/v3/users/a4498ae758ec4eb3ae945911999fbd9b => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:48.049131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f81b2d-15d3-480c-8f7b-61fb73e1dddc None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:48.049415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f81b2d-15d3-480c-8f7b-61fb73e1dddc None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:48.049692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f81b2d-15d3-480c-8f7b-61fb73e1dddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:48.066947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-c5f81b2d-15d3-480c-8f7b-61fb73e1dddc None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:48.073584 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c5f81b2d-15d3-480c-8f7b-61fb73e1dddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:48.132951 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1415/2816] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:48.144759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f81b2d-15d3-480c-8f7b-61fb73e1dddc None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:48.146190 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-c5f81b2d-15d3-480c-8f7b-61fb73e1dddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:48.167215 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d2680a4e4cd4ee3b10b0252fb146518', 'request_id': 'req-e333482b-a240-4a0b-9045-7802ffd4338f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.181418 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] The token cache is being invalidate because project 2d2680a4e4cd4ee3b10b0252fb146518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:48.181717 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.182388 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.183975 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1402/2817] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:47 2022] DELETE /identity/v3/projects/2d2680a4e4cd4ee3b10b0252fb146518 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:48.189032 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:48.190781 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:48.205638 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f81b2d-15d3-480c-8f7b-61fb73e1dddc None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:48.209556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:48.210250 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:48.210538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:48.210820 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:48.212841 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:48.226018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:48.229701 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:48.238971 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1416/2818] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:04:48.258730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333482b-a240-4a0b-9045-7802ffd4338f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:48.263526 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:48.264403 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:48.264687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:48.264959 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] PATH_INFO: `/v3/users/f0aee9238b764b2e87d65a837e347b36` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:48.266789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0aee9238b764b2e87d65a837e347b36)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:48.268734 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:48.493754 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0aee9238b764b2e87d65a837e347b36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.585730 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0aee9238b764b2e87d65a837e347b36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.586292 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0aee9238b764b2e87d65a837e347b36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.594873 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0aee9238b764b2e87d65a837e347b36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.595625 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-23e7cb62-2562-4968-a63b-4ce365a4eaa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0aee9238b764b2e87d65a837e347b36'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.613585 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1403/2819] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:04:48 2022] DELETE /identity/v3/users/f0aee9238b764b2e87d65a837e347b36 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:48.673286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:48.675042 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:48.688201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354b9c0-60f9-452d-bdfd-315bf3fedf80 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:48.737605 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:48.754341 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:48.755314 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:48.755598 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:48.757129 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] PATH_INFO: `/v3/projects/29ae24dcdaed4c73911a0ee797e9c496` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:48.757235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ae24dcdaed4c73911a0ee797e9c496)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:48.759105 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:48.881502 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ae24dcdaed4c73911a0ee797e9c496', 'request_id': 'req-afc24a02-6d3d-41eb-b92e-220e0e968e81'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:48.887160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ae24dcdaed4c73911a0ee797e9c496', 'request_id': 'req-afc24a02-6d3d-41eb-b92e-220e0e968e81'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:49.037098 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ae24dcdaed4c73911a0ee797e9c496', 'request_id': 'req-afc24a02-6d3d-41eb-b92e-220e0e968e81'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:49.050095 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] The token cache is being invalidate because project 29ae24dcdaed4c73911a0ee797e9c496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:49.050387 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:49.051047 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:49.052603 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1417/2820] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:48 2022] DELETE /identity/v3/projects/29ae24dcdaed4c73911a0ee797e9c496 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:49.057680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fddc6-5273-442e-83ab-744e2604ec69 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.058142 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fddc6-5273-442e-83ab-744e2604ec69 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.061072 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fddc6-5273-442e-83ab-744e2604ec69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.076324 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-695fddc6-5273-442e-83ab-744e2604ec69 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:49.077514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-695fddc6-5273-442e-83ab-744e2604ec69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.130059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.135651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.148637 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1404/2821] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:49.150178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a48ef-f5d2-435f-a99f-65f897d4fde7 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.150472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a48ef-f5d2-435f-a99f-65f897d4fde7 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.150783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a48ef-f5d2-435f-a99f-65f897d4fde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.158057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:49.170988 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-353a48ef-f5d2-435f-a99f-65f897d4fde7 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:49.173137 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-353a48ef-f5d2-435f-a99f-65f897d4fde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.181783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1405/2822] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:49.186210 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a48ef-f5d2-435f-a99f-65f897d4fde7 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.195665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a48ef-f5d2-435f-a99f-65f897d4fde7 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.203473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929946-5fba-49c4-8ad2-966a5a7ce1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.203473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929946-5fba-49c4-8ad2-966a5a7ce1ce admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.203473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929946-5fba-49c4-8ad2-966a5a7ce1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.203473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929946-5fba-49c4-8ad2-966a5a7ce1ce admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.204262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd929946-5fba-49c4-8ad2-966a5a7ce1ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:49.204262 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd929946-5fba-49c4-8ad2-966a5a7ce1ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.204446 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd929946-5fba-49c4-8ad2-966a5a7ce1ce admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.216244 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1406/2823] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:04:49.220350 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc24a02-6d3d-41eb-b92e-220e0e968e81 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.221122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929946-5fba-49c4-8ad2-966a5a7ce1ce admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.223164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.223861 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.224145 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.224423 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] PATH_INFO: `/v3/projects/9ed30f8779ac4acfbfa9f060a5826645` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.225630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ed30f8779ac4acfbfa9f060a5826645)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.227727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.231195 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929946-5fba-49c4-8ad2-966a5a7ce1ce admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.236929 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6b281-de15-4137-8547-bccbd115cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.237630 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b281-de15-4137-8547-bccbd115cce4 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.237944 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b281-de15-4137-8547-bccbd115cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.238219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b281-de15-4137-8547-bccbd115cce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.238895 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6b281-de15-4137-8547-bccbd115cce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.240814 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6b281-de15-4137-8547-bccbd115cce4 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.338326 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ed30f8779ac4acfbfa9f060a5826645', 'request_id': 'req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:49.344227 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ed30f8779ac4acfbfa9f060a5826645', 'request_id': 'req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:49.362304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1407/2824] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:49 2022] POST /identity/v3/projects => generated 397 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:49.366940 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6b281-de15-4137-8547-bccbd115cce4 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.412344 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ed30f8779ac4acfbfa9f060a5826645', 'request_id': 'req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:49.424661 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] The token cache is being invalidate because project 9ed30f8779ac4acfbfa9f060a5826645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:04:49.425076 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:49.426087 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-5ccb5eb2-c7f4-4006-89a5-44a9743ba5a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:04:49.427821 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1418/2825] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:04:49 2022] DELETE /identity/v3/projects/9ed30f8779ac4acfbfa9f060a5826645 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:04:49.430558 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6b281-de15-4137-8547-bccbd115cce4 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.432660 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f7f8-7f37-4b87-bce3-e3850551ed22 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.432953 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f7f8-7f37-4b87-bce3-e3850551ed22 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.433239 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f7f8-7f37-4b87-bce3-e3850551ed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.436073 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e97e6-23ff-434e-83bc-de2cc7ba3278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.436789 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e97e6-23ff-434e-83bc-de2cc7ba3278 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.437104 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e97e6-23ff-434e-83bc-de2cc7ba3278 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.437385 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e97e6-23ff-434e-83bc-de2cc7ba3278 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.438115 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e97e6-23ff-434e-83bc-de2cc7ba3278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.440105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e97e6-23ff-434e-83bc-de2cc7ba3278 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.454207 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-8ae5f7f8-7f37-4b87-bce3-e3850551ed22 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:49.455407 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-8ae5f7f8-7f37-4b87-bce3-e3850551ed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.463441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1419/2826] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:49.472681 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f807363-d9f1-494e-a4b8-6839cf0f993e None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.474266 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f807363-d9f1-494e-a4b8-6839cf0f993e None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.474601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f807363-d9f1-494e-a4b8-6839cf0f993e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.494014 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-7f807363-d9f1-494e-a4b8-6839cf0f993e None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:49.495242 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7f807363-d9f1-494e-a4b8-6839cf0f993e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.503283 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1420/2827] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:49.507489 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f807363-d9f1-494e-a4b8-6839cf0f993e None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.516862 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f807363-d9f1-494e-a4b8-6839cf0f993e None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.521432 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59127d-8b2b-4dd9-a60d-168975c7ad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.522112 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59127d-8b2b-4dd9-a60d-168975c7ad44 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.522400 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59127d-8b2b-4dd9-a60d-168975c7ad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.522692 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59127d-8b2b-4dd9-a60d-168975c7ad44 admin admin] PATH_INFO: `/v3/domains` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.523401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59127d-8b2b-4dd9-a60d-168975c7ad44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:04:49.523712 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59127d-8b2b-4dd9-a60d-168975c7ad44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.525300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59127d-8b2b-4dd9-a60d-168975c7ad44 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.537367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1421/2828] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 16:04:49.545175 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59127d-8b2b-4dd9-a60d-168975c7ad44 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.554981 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59127d-8b2b-4dd9-a60d-168975c7ad44 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.559494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e6460-0895-434b-bd2d-2f6eba9bc47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.560133 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e6460-0895-434b-bd2d-2f6eba9bc47f admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.560418 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e6460-0895-434b-bd2d-2f6eba9bc47f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.560705 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e6460-0895-434b-bd2d-2f6eba9bc47f admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.561384 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e6460-0895-434b-bd2d-2f6eba9bc47f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.563696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e6460-0895-434b-bd2d-2f6eba9bc47f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.571760 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1408/2829] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:49 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:49.572956 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e97e6-23ff-434e-83bc-de2cc7ba3278 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.574656 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-438e97e6-23ff-434e-83bc-de2cc7ba3278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.587304 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e97e6-23ff-434e-83bc-de2cc7ba3278 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.592113 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcd125-a6e3-4a0b-8361-90799ae46a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.592718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcd125-a6e3-4a0b-8361-90799ae46a1f admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.593005 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcd125-a6e3-4a0b-8361-90799ae46a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.593357 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcd125-a6e3-4a0b-8361-90799ae46a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.594619 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcd125-a6e3-4a0b-8361-90799ae46a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.596243 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcd125-a6e3-4a0b-8361-90799ae46a1f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.607680 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1409/2830] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:04:49.611689 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcd125-a6e3-4a0b-8361-90799ae46a1f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.621345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcd125-a6e3-4a0b-8361-90799ae46a1f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.626265 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313be719-808e-4429-bb20-029d8ce7d399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.629122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313be719-808e-4429-bb20-029d8ce7d399 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.629122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313be719-808e-4429-bb20-029d8ce7d399 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.629122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313be719-808e-4429-bb20-029d8ce7d399 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.629122 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313be719-808e-4429-bb20-029d8ce7d399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.629744 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313be719-808e-4429-bb20-029d8ce7d399 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.639729 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1410/2831] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:49.644282 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313be719-808e-4429-bb20-029d8ce7d399 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.653667 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313be719-808e-4429-bb20-029d8ce7d399 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.658847 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b23ef7-7d8d-4c5c-ac71-03ce1403bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.659698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b23ef7-7d8d-4c5c-ac71-03ce1403bf1c admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.659979 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b23ef7-7d8d-4c5c-ac71-03ce1403bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.660253 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b23ef7-7d8d-4c5c-ac71-03ce1403bf1c admin admin] PATH_INFO: `/v3/projects/c243fbb88e7c4c70a58d826e34b48680/users/8f5ab158cd854d81aef75adca95277fd/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.675139 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b23ef7-7d8d-4c5c-ac71-03ce1403bf1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c243fbb88e7c4c70a58d826e34b48680, user_id=8f5ab158cd854d81aef75adca95277fd, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.686464 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b23ef7-7d8d-4c5c-ac71-03ce1403bf1c admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.734720 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1422/2832] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:04:49 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:04:49.738389 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e6460-0895-434b-bd2d-2f6eba9bc47f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.800613 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e6460-0895-434b-bd2d-2f6eba9bc47f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.805289 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839bed0-24a5-4998-b4e1-7685913b7fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.805992 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839bed0-24a5-4998-b4e1-7685913b7fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.806279 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839bed0-24a5-4998-b4e1-7685913b7fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.806696 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839bed0-24a5-4998-b4e1-7685913b7fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.807396 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839bed0-24a5-4998-b4e1-7685913b7fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.809391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839bed0-24a5-4998-b4e1-7685913b7fe7 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:49.871515 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1411/2833] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:49 2022] PUT /identity/v3/projects/c243fbb88e7c4c70a58d826e34b48680/users/8f5ab158cd854d81aef75adca95277fd/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:49.882838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0e27-5557-4c6d-87c4-9f94efdf5aef None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.883131 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0e27-5557-4c6d-87c4-9f94efdf5aef None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.883414 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f0e27-5557-4c6d-87c4-9f94efdf5aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.911905 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1423/2834] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:04:49 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:04:49.917870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-5a2f0e27-5557-4c6d-87c4-9f94efdf5aef None None] MFA Rules not processed for user `8f5ab158cd854d81aef75adca95277fd`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:49.917870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5a2f0e27-5557-4c6d-87c4-9f94efdf5aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.918988 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839bed0-24a5-4998-b4e1-7685913b7fe7 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.929247 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839bed0-24a5-4998-b4e1-7685913b7fe7 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:49.981660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1412/2835] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 16:04:49.992783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f0e27-5557-4c6d-87c4-9f94efdf5aef None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:49.993039 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407fd15-3e59-4e30-bff8-97b9a0723803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:49.993759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407fd15-3e59-4e30-bff8-97b9a0723803 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:49.994072 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407fd15-3e59-4e30-bff8-97b9a0723803 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:49.994194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5a2f0e27-5557-4c6d-87c4-9f94efdf5aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:49.994353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407fd15-3e59-4e30-bff8-97b9a0723803 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:49.995320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407fd15-3e59-4e30-bff8-97b9a0723803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:49.996948 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0407fd15-3e59-4e30-bff8-97b9a0723803 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:50.009066 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1424/2836] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:50.013773 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407fd15-3e59-4e30-bff8-97b9a0723803 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:50.024151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407fd15-3e59-4e30-bff8-97b9a0723803 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:50.029080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43ec42-d17d-42a1-9231-c3194eb50890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:50.029741 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43ec42-d17d-42a1-9231-c3194eb50890 admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:50.030059 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43ec42-d17d-42a1-9231-c3194eb50890 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:50.030339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43ec42-d17d-42a1-9231-c3194eb50890 admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:50.031061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43ec42-d17d-42a1-9231-c3194eb50890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:50.032716 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43ec42-d17d-42a1-9231-c3194eb50890 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:50.046840 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1425/2837] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:04:50.051024 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43ec42-d17d-42a1-9231-c3194eb50890 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:50.061181 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43ec42-d17d-42a1-9231-c3194eb50890 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:50.061407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f0e27-5557-4c6d-87c4-9f94efdf5aef None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:50.066827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286956b6-ffc6-46ce-b601-e1bfa8a617b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:50.067725 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286956b6-ffc6-46ce-b601-e1bfa8a617b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:50.067851 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c85413-1275-4dcf-ad77-6b5d97770a1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:50.068008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286956b6-ffc6-46ce-b601-e1bfa8a617b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:50.068282 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286956b6-ffc6-46ce-b601-e1bfa8a617b8 admin admin] PATH_INFO: `/v3/projects/e759b0a0c7394efa8837a15233277f1f/users/2aceae2ec5ac4cf8b2d40a92965d7e4f/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:50.068444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c85413-1275-4dcf-ad77-6b5d97770a1a service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:50.068733 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c85413-1275-4dcf-ad77-6b5d97770a1a service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:50.069135 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c85413-1275-4dcf-ad77-6b5d97770a1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:50.078880 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c85413-1275-4dcf-ad77-6b5d97770a1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:50.080305 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c85413-1275-4dcf-ad77-6b5d97770a1a service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:50.084441 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286956b6-ffc6-46ce-b601-e1bfa8a617b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e759b0a0c7394efa8837a15233277f1f, user_id=2aceae2ec5ac4cf8b2d40a92965d7e4f, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:50.087235 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286956b6-ffc6-46ce-b601-e1bfa8a617b8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:50.089838 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1413/2838] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:04:50.186008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1426/2839] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:04:50 2022] PUT /identity/v3/projects/e759b0a0c7394efa8837a15233277f1f/users/2aceae2ec5ac4cf8b2d40a92965d7e4f/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:04:50.253442 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6d7b6-c406-4486-a4b7-253ef89c0d0f None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:50.253724 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6d7b6-c406-4486-a4b7-253ef89c0d0f None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:50.256590 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6d7b6-c406-4486-a4b7-253ef89c0d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:50.283665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-15c6d7b6-c406-4486-a4b7-253ef89c0d0f None None] MFA Rules not processed for user `2aceae2ec5ac4cf8b2d40a92965d7e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:04:50.287301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-15c6d7b6-c406-4486-a4b7-253ef89c0d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:50.311048 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:50.356896 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1414/2840] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:04:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:04:50.369318 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286956b6-ffc6-46ce-b601-e1bfa8a617b8 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:50.426651 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286956b6-ffc6-46ce-b601-e1bfa8a617b8 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:50.430437 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de56dfe-2b38-4f4e-9be7-fa6dab37f140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:50.431119 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de56dfe-2b38-4f4e-9be7-fa6dab37f140 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:50.431404 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de56dfe-2b38-4f4e-9be7-fa6dab37f140 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:50.431697 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de56dfe-2b38-4f4e-9be7-fa6dab37f140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:50.441061 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de56dfe-2b38-4f4e-9be7-fa6dab37f140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:50.442536 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de56dfe-2b38-4f4e-9be7-fa6dab37f140 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:50.451450 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1427/2841] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:04:50.671473 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:51.585393 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:04:57.028476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6d7b6-c406-4486-a4b7-253ef89c0d0f None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:04:57.039887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6d7b6-c406-4486-a4b7-253ef89c0d0f None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:04:57.044937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a5aef-38b8-44d9-91a7-ac4dc96e8867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:04:57.045616 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5aef-38b8-44d9-91a7-ac4dc96e8867 service cinder] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:04:57.046203 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5aef-38b8-44d9-91a7-ac4dc96e8867 service cinder] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:04:57.046980 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a5aef-38b8-44d9-91a7-ac4dc96e8867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:04:57.049064 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-197a5aef-38b8-44d9-91a7-ac4dc96e8867 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:04:57.119320 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a5aef-38b8-44d9-91a7-ac4dc96e8867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:04:57.120871 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a5aef-38b8-44d9-91a7-ac4dc96e8867 service cinder] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:04:57.133208 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1415/2842] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:04:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:05:01.967975 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=22 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:07.062802 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de56dfe-2b38-4f4e-9be7-fa6dab37f140 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:07.125798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de56dfe-2b38-4f4e-9be7-fa6dab37f140 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:07.127007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:07.130514 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:07.131383 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:07.131666 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:07.131974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] PATH_INFO: `/v3/users/8f5ab158cd854d81aef75adca95277fd` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:07.133152 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f5ab158cd854d81aef75adca95277fd)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:07.135137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:07.416485 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f5ab158cd854d81aef75adca95277fd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:07.487511 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f5ab158cd854d81aef75adca95277fd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:07.488142 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f5ab158cd854d81aef75adca95277fd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:07.496884 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f5ab158cd854d81aef75adca95277fd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:07.497567 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f5ab158cd854d81aef75adca95277fd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:07.516798 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1428/2843] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:05:07 2022] DELETE /identity/v3/users/8f5ab158cd854d81aef75adca95277fd => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:05:07.521417 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e824f4e-ab1e-43ca-a25f-3d885d2cda72 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:07.521706 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e824f4e-ab1e-43ca-a25f-3d885d2cda72 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:07.522027 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e824f4e-ab1e-43ca-a25f-3d885d2cda72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:07.539748 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-2e824f4e-ab1e-43ca-a25f-3d885d2cda72 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:05:07.549918 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2e824f4e-ab1e-43ca-a25f-3d885d2cda72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:07.607628 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1416/2844] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:05:07.619966 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:07.621542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:07.687693 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6664b7-17ed-453e-8cd0-0421577d56bf admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:07.691740 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ddbae-8d39-4421-8ab1-d5180d2a242c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:07.692415 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ddbae-8d39-4421-8ab1-d5180d2a242c service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:07.692698 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ddbae-8d39-4421-8ab1-d5180d2a242c service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:07.692976 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ddbae-8d39-4421-8ab1-d5180d2a242c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:07.703597 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ddbae-8d39-4421-8ab1-d5180d2a242c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:07.705055 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ddbae-8d39-4421-8ab1-d5180d2a242c service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:07.715868 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1429/2845] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:05:07.937144 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc8f53-cd1c-4af2-9360-12ca3e42c475 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:07.937883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc8f53-cd1c-4af2-9360-12ca3e42c475 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:07.937883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc8f53-cd1c-4af2-9360-12ca3e42c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:07.959954 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-7dcc8f53-cd1c-4af2-9360-12ca3e42c475 None None] MFA Rules not processed for user `ba765c1a8c6c449ca699eb7a44dde50b`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:05:07.961436 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7dcc8f53-cd1c-4af2-9360-12ca3e42c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:08.029310 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1417/2846] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:05:08.076571 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ddbae-8d39-4421-8ab1-d5180d2a242c service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:08.081080 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-232ddbae-8d39-4421-8ab1-d5180d2a242c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:08.111601 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:08.158831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ddbae-8d39-4421-8ab1-d5180d2a242c service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:08.163065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48ea1e-a666-44b3-814e-d254e5685041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d54d684efce44b99f3235e80beb1c56', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:08.163909 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48ea1e-a666-44b3-814e-d254e5685041 service cinder] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:08.164300 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48ea1e-a666-44b3-814e-d254e5685041 service cinder] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:08.164680 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48ea1e-a666-44b3-814e-d254e5685041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:08.167429 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ce48ea1e-a666-44b3-814e-d254e5685041 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:08.243571 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc8f53-cd1c-4af2-9360-12ca3e42c475 None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:08.245185 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-7dcc8f53-cd1c-4af2-9360-12ca3e42c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:08.251600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48ea1e-a666-44b3-814e-d254e5685041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:08.253742 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48ea1e-a666-44b3-814e-d254e5685041 service cinder] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:08.260435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc8f53-cd1c-4af2-9360-12ca3e42c475 None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:08.266174 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:08.267146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:08.267391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:08.267665 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] PATH_INFO: `/v3/projects/c243fbb88e7c4c70a58d826e34b48680` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:08.269468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c243fbb88e7c4c70a58d826e34b48680)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:08.271136 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:08.274768 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1430/2847] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:05:08.445271 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c243fbb88e7c4c70a58d826e34b48680', 'request_id': 'req-2b85f1fe-9c97-4417-9df7-2a91f619cedd'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:08.451374 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c243fbb88e7c4c70a58d826e34b48680', 'request_id': 'req-2b85f1fe-9c97-4417-9df7-2a91f619cedd'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:08.526875 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c243fbb88e7c4c70a58d826e34b48680', 'request_id': 'req-2b85f1fe-9c97-4417-9df7-2a91f619cedd'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:08.539293 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] The token cache is being invalidate because project c243fbb88e7c4c70a58d826e34b48680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:05:08.539722 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:08.540407 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:08.542105 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1418/2848] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:05:08 2022] DELETE /identity/v3/projects/c243fbb88e7c4c70a58d826e34b48680 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:05:09.545630 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48ea1e-a666-44b3-814e-d254e5685041 service cinder] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:09.556480 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48ea1e-a666-44b3-814e-d254e5685041 service cinder] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:09.610118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb64618-b899-4e1f-8354-7d7e07f77ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:09.610782 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb64618-b899-4e1f-8354-7d7e07f77ede service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:09.611074 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb64618-b899-4e1f-8354-7d7e07f77ede service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:09.611353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb64618-b899-4e1f-8354-7d7e07f77ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:09.613267 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fcb64618-b899-4e1f-8354-7d7e07f77ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:09.693492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb64618-b899-4e1f-8354-7d7e07f77ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:09.694993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb64618-b899-4e1f-8354-7d7e07f77ede service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:09.707913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1431/2849] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:05:18.535993 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:19.389955 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:19.391362 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:19.455478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b85f1fe-9c97-4417-9df7-2a91f619cedd admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:19.459973 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:19.460840 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:19.461118 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:19.461391 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] PATH_INFO: `/v3/users/2aceae2ec5ac4cf8b2d40a92965d7e4f` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:19.462583 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aceae2ec5ac4cf8b2d40a92965d7e4f)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:19.464491 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:19.650232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aceae2ec5ac4cf8b2d40a92965d7e4f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:19.704025 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=98 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:19.732795 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aceae2ec5ac4cf8b2d40a92965d7e4f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:19.733435 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aceae2ec5ac4cf8b2d40a92965d7e4f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:19.740932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aceae2ec5ac4cf8b2d40a92965d7e4f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:19.741560 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aceae2ec5ac4cf8b2d40a92965d7e4f'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:19.759296 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1419/2850] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:05:19 2022] DELETE /identity/v3/users/2aceae2ec5ac4cf8b2d40a92965d7e4f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:05:19.763940 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec970c88-6340-48ef-bec0-b1e74cecd62b None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:19.764221 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec970c88-6340-48ef-bec0-b1e74cecd62b None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:19.765244 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec970c88-6340-48ef-bec0-b1e74cecd62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:19.782088 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-ec970c88-6340-48ef-bec0-b1e74cecd62b None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:05:19.783306 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ec970c88-6340-48ef-bec0-b1e74cecd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:19.842887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1432/2851] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:05:19.852699 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:19.854181 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:19.912246 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20835692-6b0e-4d30-a157-4549bcd249ca admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:19.915736 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65c440-cf76-49c6-ac33-d5f3361f641a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:19.916384 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c440-cf76-49c6-ac33-d5f3361f641a service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:19.916663 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c440-cf76-49c6-ac33-d5f3361f641a service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:19.916937 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c440-cf76-49c6-ac33-d5f3361f641a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:19.926232 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65c440-cf76-49c6-ac33-d5f3361f641a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:19.927650 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65c440-cf76-49c6-ac33-d5f3361f641a service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:19.937502 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1420/2852] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:05:20.483750 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec970c88-6340-48ef-bec0-b1e74cecd62b None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:20.486347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ec970c88-6340-48ef-bec0-b1e74cecd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:20.502419 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec970c88-6340-48ef-bec0-b1e74cecd62b None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:20.511189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:20.511189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:20.511189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:20.511189 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] PATH_INFO: `/v3/projects/e759b0a0c7394efa8837a15233277f1f` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:20.512044 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e759b0a0c7394efa8837a15233277f1f)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:20.524118 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:20.675443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e759b0a0c7394efa8837a15233277f1f', 'request_id': 'req-84606cf5-5a8f-4d6d-9844-46f362c806a2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:20.681037 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e759b0a0c7394efa8837a15233277f1f', 'request_id': 'req-84606cf5-5a8f-4d6d-9844-46f362c806a2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:20.767573 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e759b0a0c7394efa8837a15233277f1f', 'request_id': 'req-84606cf5-5a8f-4d6d-9844-46f362c806a2'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:20.779467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] The token cache is being invalidate because project e759b0a0c7394efa8837a15233277f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:05:20.779749 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:20.780427 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-84606cf5-5a8f-4d6d-9844-46f362c806a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:05:20.782277 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1433/2853] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:05:20 2022] DELETE /identity/v3/projects/e759b0a0c7394efa8837a15233277f1f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:05:20.787234 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7f114-2dcd-4aeb-9461-f15deedfaead None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:20.787774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7f114-2dcd-4aeb-9461-f15deedfaead None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:20.787774 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7f114-2dcd-4aeb-9461-f15deedfaead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:20.805798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-1ad7f114-2dcd-4aeb-9461-f15deedfaead None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:05:20.809686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-1ad7f114-2dcd-4aeb-9461-f15deedfaead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:20.826052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:20.869687 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1421/2854] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:05:20.878447 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16525c8-d6df-4d97-bcc1-1a744180d034 None None] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:20.878735 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16525c8-d6df-4d97-bcc1-1a744180d034 None None] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:20.879065 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16525c8-d6df-4d97-bcc1-1a744180d034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:20.899668 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.auth.core [None req-d16525c8-d6df-4d97-bcc1-1a744180d034 None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:05:20.901494 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-d16525c8-d6df-4d97-bcc1-1a744180d034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:20.908225 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1434/2855] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:05:20.912545 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7f114-2dcd-4aeb-9461-f15deedfaead None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:20.925126 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7f114-2dcd-4aeb-9461-f15deedfaead None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:20.930317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b961fe-14d8-4fad-9235-394761dca1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:20.930982 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b961fe-14d8-4fad-9235-394761dca1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:20.931266 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b961fe-14d8-4fad-9235-394761dca1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:20.931541 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b961fe-14d8-4fad-9235-394761dca1b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:20.932264 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b961fe-14d8-4fad-9235-394761dca1b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 16:05:20.932569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b961fe-14d8-4fad-9235-394761dca1b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:20.934194 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b961fe-14d8-4fad-9235-394761dca1b3 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:20.947932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1422/2856] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 14 16:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 16:05:20.952353 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16525c8-d6df-4d97-bcc1-1a744180d034 None None] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:20.962075 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16525c8-d6df-4d97-bcc1-1a744180d034 None None] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:20.966926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a1691-86d3-4f8f-9ed1-1ece00cd6516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:20.967594 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a1691-86d3-4f8f-9ed1-1ece00cd6516 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:20.967876 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a1691-86d3-4f8f-9ed1-1ece00cd6516 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:20.968151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a1691-86d3-4f8f-9ed1-1ece00cd6516 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:20.969000 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a1691-86d3-4f8f-9ed1-1ece00cd6516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:20.970922 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a1691-86d3-4f8f-9ed1-1ece00cd6516 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:21.106294 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1435/2857] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:05:20 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:05:21.111856 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b961fe-14d8-4fad-9235-394761dca1b3 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:21.122177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b961fe-14d8-4fad-9235-394761dca1b3 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:21.183651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c961b1-d58e-4410-ab80-02a9e4a9f505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:21.184398 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c961b1-d58e-4410-ab80-02a9e4a9f505 admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:21.184712 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c961b1-d58e-4410-ab80-02a9e4a9f505 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:21.184984 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c961b1-d58e-4410-ab80-02a9e4a9f505 admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:21.185705 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c961b1-d58e-4410-ab80-02a9e4a9f505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:21.188673 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c961b1-d58e-4410-ab80-02a9e4a9f505 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:21.317782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1423/2858] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:05:21 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:05:21.320762 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a1691-86d3-4f8f-9ed1-1ece00cd6516 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:21.331769 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a1691-86d3-4f8f-9ed1-1ece00cd6516 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:21.336542 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9151b-b74b-4a50-811d-58241e6f8b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:21.337224 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9151b-b74b-4a50-811d-58241e6f8b0d admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:21.337512 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9151b-b74b-4a50-811d-58241e6f8b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:21.337794 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9151b-b74b-4a50-811d-58241e6f8b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:21.338575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9151b-b74b-4a50-811d-58241e6f8b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:21.340214 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9151b-b74b-4a50-811d-58241e6f8b0d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:21.351933 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1436/2859] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:05:21.355870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c961b1-d58e-4410-ab80-02a9e4a9f505 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:21.370497 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c961b1-d58e-4410-ab80-02a9e4a9f505 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:21.375286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340b506-5d4a-4a81-a4d3-31aa56bd6325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:21.375951 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340b506-5d4a-4a81-a4d3-31aa56bd6325 admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:21.376231 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340b506-5d4a-4a81-a4d3-31aa56bd6325 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:21.376506 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340b506-5d4a-4a81-a4d3-31aa56bd6325 admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:21.377226 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340b506-5d4a-4a81-a4d3-31aa56bd6325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:21.379013 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340b506-5d4a-4a81-a4d3-31aa56bd6325 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:21.389483 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1424/2860] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:05:21.393774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9151b-b74b-4a50-811d-58241e6f8b0d admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:21.404134 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9151b-b74b-4a50-811d-58241e6f8b0d admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:21.409524 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e11f02-b266-4fb0-af05-523b3fa40c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:21.410456 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e11f02-b266-4fb0-af05-523b3fa40c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:21.414251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e11f02-b266-4fb0-af05-523b3fa40c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:21.414251 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e11f02-b266-4fb0-af05-523b3fa40c2f admin admin] PATH_INFO: `/v3/projects/955b9d20401546f9a3f8477a9e67fc24/users/0d16fe0fde164dd8b0652168fc438263/roles/fda37aea043b4685b930f1d9bde2b0ec` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:21.426774 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e11f02-b266-4fb0-af05-523b3fa40c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=955b9d20401546f9a3f8477a9e67fc24, user_id=0d16fe0fde164dd8b0652168fc438263, role_id=fda37aea043b4685b930f1d9bde2b0ec)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:21.429458 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e11f02-b266-4fb0-af05-523b3fa40c2f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:21.509228 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1437/2861] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:05:21 2022] PUT /identity/v3/projects/955b9d20401546f9a3f8477a9e67fc24/users/0d16fe0fde164dd8b0652168fc438263/roles/fda37aea043b4685b930f1d9bde2b0ec => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:05:21.514770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1574f17-d7d9-4b89-9c86-cc53405d075a None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:21.515058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1574f17-d7d9-4b89-9c86-cc53405d075a None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:21.515337 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1574f17-d7d9-4b89-9c86-cc53405d075a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:21.533990 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-e1574f17-d7d9-4b89-9c86-cc53405d075a None None] MFA Rules not processed for user `d2ffbc8434d34115805dcbc2a72dc671`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:05:21.535239 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e1574f17-d7d9-4b89-9c86-cc53405d075a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:21.607283 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1425/2862] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:05:21.617984 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e11f02-b266-4fb0-af05-523b3fa40c2f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:21.619717 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-53e11f02-b266-4fb0-af05-523b3fa40c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:21.633543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:21.691131 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e11f02-b266-4fb0-af05-523b3fa40c2f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:21.694827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40a888-4303-4d60-aafe-51acc9bfe78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:21.695713 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40a888-4303-4d60-aafe-51acc9bfe78f service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:21.695995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40a888-4303-4d60-aafe-51acc9bfe78f service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:21.696273 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40a888-4303-4d60-aafe-51acc9bfe78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:21.709161 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40a888-4303-4d60-aafe-51acc9bfe78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:21.710623 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40a888-4303-4d60-aafe-51acc9bfe78f service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:21.721033 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1438/2863] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:05:27.901057 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a8b00-1739-448c-af43-7fce813eb2ca None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:27.901555 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a8b00-1739-448c-af43-7fce813eb2ca None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:27.901676 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a8b00-1739-448c-af43-7fce813eb2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:27.932127 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-ad6a8b00-1739-448c-af43-7fce813eb2ca None None] MFA Rules not processed for user `0d16fe0fde164dd8b0652168fc438263`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:05:27.933345 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-ad6a8b00-1739-448c-af43-7fce813eb2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:27.956576 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:28.021922 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1426/2864] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:05:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:05:28.026438 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40a888-4303-4d60-aafe-51acc9bfe78f service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:28.039167 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40a888-4303-4d60-aafe-51acc9bfe78f service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:28.044610 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eddf02-10a0-4c8a-8795-3b9f67840e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:28.044787 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eddf02-10a0-4c8a-8795-3b9f67840e25 admin admin] REQUEST_METHOD: `POST` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:28.045045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eddf02-10a0-4c8a-8795-3b9f67840e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:28.045614 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eddf02-10a0-4c8a-8795-3b9f67840e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:28.046406 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eddf02-10a0-4c8a-8795-3b9f67840e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:28.048347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eddf02-10a0-4c8a-8795-3b9f67840e25 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:28.143396 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1439/2865] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 14 16:05:28 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 16:05:28.144648 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a8b00-1739-448c-af43-7fce813eb2ca None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:28.156106 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a8b00-1739-448c-af43-7fce813eb2ca None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:28.222723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7834a-06ea-460d-829b-0a2115c2f31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:28.223668 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7834a-06ea-460d-829b-0a2115c2f31e admin admin] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:28.224029 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7834a-06ea-460d-829b-0a2115c2f31e admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:28.224379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7834a-06ea-460d-829b-0a2115c2f31e admin admin] PATH_INFO: `/v3/users` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:28.225156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7834a-06ea-460d-829b-0a2115c2f31e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:28.227214 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7834a-06ea-460d-829b-0a2115c2f31e admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:28.330998 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1427/2866] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 14 16:05:28 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 16:05:28.332153 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eddf02-10a0-4c8a-8795-3b9f67840e25 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:28.343286 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eddf02-10a0-4c8a-8795-3b9f67840e25 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:28.347887 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d285ab-3573-4a74-8ccc-0b256824264f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:28.353468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d285ab-3573-4a74-8ccc-0b256824264f admin admin] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:28.353468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d285ab-3573-4a74-8ccc-0b256824264f admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:28.353468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d285ab-3573-4a74-8ccc-0b256824264f admin admin] PATH_INFO: `/v3/roles` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:28.353468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d285ab-3573-4a74-8ccc-0b256824264f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:28.353468 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d285ab-3573-4a74-8ccc-0b256824264f admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:28.362917 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1440/2867] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 16:05:28.367178 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7834a-06ea-460d-829b-0a2115c2f31e admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:28.377432 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7834a-06ea-460d-829b-0a2115c2f31e admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:28.382001 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78515452-0f6f-4878-9e32-0f68a577a6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:28.382882 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78515452-0f6f-4878-9e32-0f68a577a6a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:28.383167 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78515452-0f6f-4878-9e32-0f68a577a6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:28.383444 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78515452-0f6f-4878-9e32-0f68a577a6a8 admin admin] PATH_INFO: `/v3/projects/454b2afb06404b2db5e42cceb572e3ac/users/15507a71243d4055a6f01ae36e5743e4/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:28.399039 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78515452-0f6f-4878-9e32-0f68a577a6a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454b2afb06404b2db5e42cceb572e3ac, user_id=15507a71243d4055a6f01ae36e5743e4, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:28.401686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78515452-0f6f-4878-9e32-0f68a577a6a8 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:28.489879 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d285ab-3573-4a74-8ccc-0b256824264f admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:28.491783 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-21d285ab-3573-4a74-8ccc-0b256824264f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:28.510371 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1428/2868] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:05:28 2022] PUT /identity/v3/projects/454b2afb06404b2db5e42cceb572e3ac/users/15507a71243d4055a6f01ae36e5743e4/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:05:28.515058 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78515452-0f6f-4878-9e32-0f68a577a6a8 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:28.558174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d285ab-3573-4a74-8ccc-0b256824264f admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:28.589449 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78515452-0f6f-4878-9e32-0f68a577a6a8 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:28.595883 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d3e2-ee12-4f1b-a485-ec5f5f06157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:28.596892 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d3e2-ee12-4f1b-a485-ec5f5f06157f admin admin] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:28.597278 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d3e2-ee12-4f1b-a485-ec5f5f06157f admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:28.597623 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d3e2-ee12-4f1b-a485-ec5f5f06157f admin admin] PATH_INFO: `/v3/roles` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:28.598463 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348d3e2-ee12-4f1b-a485-ec5f5f06157f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:28.600162 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348d3e2-ee12-4f1b-a485-ec5f5f06157f admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:28.617911 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1429/2869] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 14 16:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 16:05:28.622981 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d3e2-ee12-4f1b-a485-ec5f5f06157f admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:28.627421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09e078-e873-4dad-bdc8-cdf39537ddf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:28.628256 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09e078-e873-4dad-bdc8-cdf39537ddf2 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:28.628558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09e078-e873-4dad-bdc8-cdf39537ddf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:28.628916 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09e078-e873-4dad-bdc8-cdf39537ddf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:28.631538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ae09e078-e873-4dad-bdc8-cdf39537ddf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:28.633593 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d3e2-ee12-4f1b-a485-ec5f5f06157f admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:28.638611 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff1dd9-dd24-48a2-8220-ac80f6a21653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:28.640044 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1dd9-dd24-48a2-8220-ac80f6a21653 admin admin] REQUEST_METHOD: `PUT` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:28.640330 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1dd9-dd24-48a2-8220-ac80f6a21653 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:28.640609 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff1dd9-dd24-48a2-8220-ac80f6a21653 admin admin] PATH_INFO: `/v3/projects/454b2afb06404b2db5e42cceb572e3ac/users/15507a71243d4055a6f01ae36e5743e4/roles/abfb60dccca7470295258595a66a5d17` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:28.642960 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff1dd9-dd24-48a2-8220-ac80f6a21653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=454b2afb06404b2db5e42cceb572e3ac, user_id=15507a71243d4055a6f01ae36e5743e4, role_id=abfb60dccca7470295258595a66a5d17)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:28.643320 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09e078-e873-4dad-bdc8-cdf39537ddf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:28.645015 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09e078-e873-4dad-bdc8-cdf39537ddf2 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:28.645869 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff1dd9-dd24-48a2-8220-ac80f6a21653 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:28.658187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1441/2870] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 16:05:28.662336 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1430/2871] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 14 16:05:28 2022] PUT /identity/v3/projects/454b2afb06404b2db5e42cceb572e3ac/users/15507a71243d4055a6f01ae36e5743e4/roles/abfb60dccca7470295258595a66a5d17 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:05:31.523136 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09e078-e873-4dad-bdc8-cdf39537ddf2 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:31.534448 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09e078-e873-4dad-bdc8-cdf39537ddf2 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:31.583497 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:31.597543 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:31.598259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:31.598550 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:31.598898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:31.602650 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:31.734885 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:31.736558 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:31.753653 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1442/2872] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:05:36.203015 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77b44c-6bca-4ac3-9e3c-613961a1c0e1 None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:36.203514 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77b44c-6bca-4ac3-9e3c-613961a1c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:36.203694 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77b44c-6bca-4ac3-9e3c-613961a1c0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:36.221052 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:36.237967 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-2b77b44c-6bca-4ac3-9e3c-613961a1c0e1 None None] MFA Rules not processed for user `15507a71243d4055a6f01ae36e5743e4`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:05:36.239467 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-2b77b44c-6bca-4ac3-9e3c-613961a1c0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:36.331092 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1431/2873] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:05:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:05:36.344836 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:36.346446 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:36.430538 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74043e8-f913-49b1-ac4a-647e8f0a4100 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:36.436839 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3597f-8785-4231-94e0-a4ef8886c224 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:36.437521 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3597f-8785-4231-94e0-a4ef8886c224 service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:36.437838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3597f-8785-4231-94e0-a4ef8886c224 service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:36.438191 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3597f-8785-4231-94e0-a4ef8886c224 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:36.498736 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3597f-8785-4231-94e0-a4ef8886c224 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:36.500401 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3597f-8785-4231-94e0-a4ef8886c224 service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:36.511528 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1443/2874] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:05:39.431671 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075faad9-3aeb-4594-87d9-69097c0ca39c None None] REQUEST_METHOD: `POST` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:39.432268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075faad9-3aeb-4594-87d9-69097c0ca39c None None] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:39.432268 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075faad9-3aeb-4594-87d9-69097c0ca39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:39.453176 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.auth.core [None req-075faad9-3aeb-4594-87d9-69097c0ca39c None None] MFA Rules not processed for user `0d16fe0fde164dd8b0652168fc438263`. Rule list: `[]` (Enabled: `True`). {{(pid=92133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 16:05:39.454452 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-075faad9-3aeb-4594-87d9-69097c0ca39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:39.461747 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1432/2875] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 14 16:05:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 16:05:46.505379 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=88 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:49.445602 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=34 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:05:56.308237 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3597f-8785-4231-94e0-a4ef8886c224 service nova] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:56.318395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3597f-8785-4231-94e0-a4ef8886c224 service nova] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:56.321985 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0406b-8e47-42ff-9b51-dd4812379da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:56.322636 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0406b-8e47-42ff-9b51-dd4812379da7 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:56.322926 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0406b-8e47-42ff-9b51-dd4812379da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:56.323203 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0406b-8e47-42ff-9b51-dd4812379da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:56.324993 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-a6c0406b-8e47-42ff-9b51-dd4812379da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:56.335018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0406b-8e47-42ff-9b51-dd4812379da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:56.336467 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0406b-8e47-42ff-9b51-dd4812379da7 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:56.345338 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1444/2876] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:05:57.273472 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075faad9-3aeb-4594-87d9-69097c0ca39c None None] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:05:57.275537 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-075faad9-3aeb-4594-87d9-69097c0ca39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:57.346857 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075faad9-3aeb-4594-87d9-69097c0ca39c None None] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:05:57.357434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04f91-55ca-4ede-993a-1e2433a36788 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:05:57.357434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04f91-55ca-4ede-993a-1e2433a36788 service placement] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:05:57.357434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04f91-55ca-4ede-993a-1e2433a36788 service placement] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:05:57.357434 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a04f91-55ca-4ede-993a-1e2433a36788 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:05:57.357884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-c5a04f91-55ca-4ede-993a-1e2433a36788 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:05:57.366254 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a04f91-55ca-4ede-993a-1e2433a36788 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:05:57.367819 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a04f91-55ca-4ede-993a-1e2433a36788 service placement] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:05:57.379184 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1433/2877] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:05:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:06:01.334347 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0406b-8e47-42ff-9b51-dd4812379da7 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:01.356259 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0406b-8e47-42ff-9b51-dd4812379da7 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:01.362431 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a883213-c932-40d6-8c9b-1a981f093a8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:01.363137 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a883213-c932-40d6-8c9b-1a981f093a8a service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:01.363421 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a883213-c932-40d6-8c9b-1a981f093a8a service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:01.363899 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a883213-c932-40d6-8c9b-1a981f093a8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:01.366253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-7a883213-c932-40d6-8c9b-1a981f093a8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:06:01.381018 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a883213-c932-40d6-8c9b-1a981f093a8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:01.381532 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a883213-c932-40d6-8c9b-1a981f093a8a service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:01.394253 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1445/2878] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:06:07.374651 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=32 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:11.388669 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=12 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:06:15.357887 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04f91-55ca-4ede-993a-1e2433a36788 service placement] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:06:15.367468 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a04f91-55ca-4ede-993a-1e2433a36788 service placement] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:06:15.371000 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406edb-6b8c-4487-bfcc-8a2f3eeb2403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:06:15.371844 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406edb-6b8c-4487-bfcc-8a2f3eeb2403 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:06:15.372169 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406edb-6b8c-4487-bfcc-8a2f3eeb2403 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:06:15.372447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13406edb-6b8c-4487-bfcc-8a2f3eeb2403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:06:15.381764 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13406edb-6b8c-4487-bfcc-8a2f3eeb2403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:06:15.383257 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13406edb-6b8c-4487-bfcc-8a2f3eeb2403 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:06:15.392901 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1434/2879] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:06:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:06:25.388900 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=6 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:07:12.687890 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a883213-c932-40d6-8c9b-1a981f093a8a service nova] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:07:12.701174 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a883213-c932-40d6-8c9b-1a981f093a8a service nova] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:07:12.707268 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:07:12.708012 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:07:12.708883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:07:12.708883 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:07:12.710867 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:07:12.725281 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:07:12.727391 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:07:12.740395 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1446/2880] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:07:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:07:18.999946 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406edb-6b8c-4487-bfcc-8a2f3eeb2403 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:07:19.008858 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13406edb-6b8c-4487-bfcc-8a2f3eeb2403 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:07:19.013116 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5dd8-cc74-40a6-8973-cb5fdde6bbeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:07:19.013853 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5dd8-cc74-40a6-8973-cb5fdde6bbeb service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:07:19.014146 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5dd8-cc74-40a6-8973-cb5fdde6bbeb service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:07:19.014430 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5dd8-cc74-40a6-8973-cb5fdde6bbeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:07:19.016237 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-b6ee5dd8-cc74-40a6-8973-cb5fdde6bbeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:07:19.084952 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5dd8-cc74-40a6-8973-cb5fdde6bbeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:07:19.086447 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5dd8-cc74-40a6-8973-cb5fdde6bbeb service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:07:19.096348 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1435/2881] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:07:22.734455 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=6 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:07:26.765287 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:07:26.767083 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:07:26.782367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9155bc-4061-435e-ba71-25e2b5017efd service nova] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:07:26.787272 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:07:26.788158 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:07:26.788443 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:07:26.788721 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] PATH_INFO: `/v3/users/ba765c1a8c6c449ca699eb7a44dde50b` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:07:26.799710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba765c1a8c6c449ca699eb7a44dde50b)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:07:26.799710 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:07:27.036117 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba765c1a8c6c449ca699eb7a44dde50b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.117554 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba765c1a8c6c449ca699eb7a44dde50b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.118187 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba765c1a8c6c449ca699eb7a44dde50b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.126106 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba765c1a8c6c449ca699eb7a44dde50b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.126803 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba765c1a8c6c449ca699eb7a44dde50b'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.145490 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1447/2882] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:07:26 2022] DELETE /identity/v3/users/ba765c1a8c6c449ca699eb7a44dde50b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:07:27.388783 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5dd8-cc74-40a6-8973-cb5fdde6bbeb service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:07:27.401437 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5dd8-cc74-40a6-8973-cb5fdde6bbeb service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:07:27.463159 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:07:27.464053 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:07:27.464340 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:07:27.464618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] PATH_INFO: `/v3/projects/93233d8a364e44edad9877e1eaa82b4d` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:07:27.465842 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] RBAC: Authorizing `identity:delete_project(project_id=93233d8a364e44edad9877e1eaa82b4d)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:07:27.468380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:07:27.581393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93233d8a364e44edad9877e1eaa82b4d', 'request_id': 'req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.586992 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93233d8a364e44edad9877e1eaa82b4d', 'request_id': 'req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.683907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93233d8a364e44edad9877e1eaa82b4d', 'request_id': 'req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.695618 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] The token cache is being invalidate because project 93233d8a364e44edad9877e1eaa82b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:07:27.695907 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.696569 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:27.698124 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1436/2883] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:07:27 2022] DELETE /identity/v3/projects/93233d8a364e44edad9877e1eaa82b4d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:07:27.702322 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:07:27.703827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:07:27.717445 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b3a96-bd95-4975-a82b-bc32710a37f3 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:07:27.778852 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:07:27.779727 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:07:27.780008 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:07:27.780284 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] PATH_INFO: `/v3/users/a22d987d7e914bdaa018d90232f12bdd` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:07:27.781600 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22d987d7e914bdaa018d90232f12bdd)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:07:27.783557 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:07:27.970160 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22d987d7e914bdaa018d90232f12bdd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.047913 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22d987d7e914bdaa018d90232f12bdd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.048572 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22d987d7e914bdaa018d90232f12bdd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.056555 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22d987d7e914bdaa018d90232f12bdd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.057492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22d987d7e914bdaa018d90232f12bdd'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.065968 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:07:28.076327 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1448/2884] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:07:27 2022] DELETE /identity/v3/users/a22d987d7e914bdaa018d90232f12bdd => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:07:28.331870 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:07:28.333401 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:07:28.346614 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fbbf5-6173-42dd-b0d2-e5dadf28003b admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:07:28.368267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:07:28.407248 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:07:28.408192 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:07:28.408698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:07:28.409025 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] PATH_INFO: `/v3/projects/cf8abc30c0cd47c1b6b1c2d35b8f7f9d` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:07:28.410741 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf8abc30c0cd47c1b6b1c2d35b8f7f9d)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:07:28.412686 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:07:28.543066 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf8abc30c0cd47c1b6b1c2d35b8f7f9d', 'request_id': 'req-a509ae08-7dd5-496a-88b8-bae28bed33a2'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.548633 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf8abc30c0cd47c1b6b1c2d35b8f7f9d', 'request_id': 'req-a509ae08-7dd5-496a-88b8-bae28bed33a2'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.638024 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf8abc30c0cd47c1b6b1c2d35b8f7f9d', 'request_id': 'req-a509ae08-7dd5-496a-88b8-bae28bed33a2'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.649751 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] The token cache is being invalidate because project cf8abc30c0cd47c1b6b1c2d35b8f7f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:07:28.650075 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.650762 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:07:28.652267 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1437/2885] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:07:28 2022] DELETE /identity/v3/projects/cf8abc30c0cd47c1b6b1c2d35b8f7f9d => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 16:07:38.180624 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:07:38.646770 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:08:28.283125 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:08:28.284995 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:28.378871 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48325b31-aa73-4d6c-b4db-a5021201f827 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:08:28.461675 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc31676-caa1-4452-a08d-c1bae894e931 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:08:28.463303 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc31676-caa1-4452-a08d-c1bae894e931 service placement] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:08:28.463759 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc31676-caa1-4452-a08d-c1bae894e931 service placement] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:08:28.464205 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc31676-caa1-4452-a08d-c1bae894e931 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:08:28.467120 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ffc31676-caa1-4452-a08d-c1bae894e931 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:28.485316 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc31676-caa1-4452-a08d-c1bae894e931 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:08:28.486979 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc31676-caa1-4452-a08d-c1bae894e931 service placement] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:08:28.498974 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1449/2886] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:08:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:08:29.813674 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:08:29.815180 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:29.881365 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509ae08-7dd5-496a-88b8-bae28bed33a2 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:08:29.888329 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:08:29.888931 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:08:29.889219 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:08:29.889501 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:08:29.891380 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:29.903792 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:08:29.905331 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:08:29.914921 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1438/2887] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:08:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:08:30.232685 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc31676-caa1-4452-a08d-c1bae894e931 service placement] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:08:30.243151 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc31676-caa1-4452-a08d-c1bae894e931 service placement] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:08:30.248173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:08:30.249086 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:08:30.249505 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:08:30.249947 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:08:30.253746 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:30.266831 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:08:30.268576 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:08:30.278939 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1450/2888] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:08:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:08:30.378487 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:08:30.380207 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:30.399003 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027001a-115d-4085-81b7-6ed823f013b9 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:08:30.404779 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc85aa-4e28-4fa5-9b72-514e6a2e91c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:08:30.405440 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc85aa-4e28-4fa5-9b72-514e6a2e91c3 service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:08:30.405718 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc85aa-4e28-4fa5-9b72-514e6a2e91c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:08:30.406026 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc85aa-4e28-4fa5-9b72-514e6a2e91c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:08:30.407803 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-13dc85aa-4e28-4fa5-9b72-514e6a2e91c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:30.481078 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc85aa-4e28-4fa5-9b72-514e6a2e91c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:08:30.483660 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc85aa-4e28-4fa5-9b72-514e6a2e91c3 service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:08:30.499763 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1439/2889] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:08:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:08:30.764912 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:08:30.766363 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:30.787531 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0a48f-676a-4943-b4f7-da0ddf50bf12 service neutron] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:08:30.795463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce88c973feef4627a1b871ee9cb26511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:08:30.796123 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:08:30.796556 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:08:30.796838 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:08:30.798780 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:08:30.812382 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:08:30.814989 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:08:30.829694 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1451/2890] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:08:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:08:40.494298 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=60 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:08:40.821367 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=70 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:10:09.029989 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc85aa-4e28-4fa5-9b72-514e6a2e91c3 service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:09.039186 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc85aa-4e28-4fa5-9b72-514e6a2e91c3 service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:09.043478 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e63b8e907764f2c818391ab2ad9184c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:09.046620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:09.046620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:09.046620 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:09.047476 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:09.061147 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:09.062643 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:09.074765 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1440/2891] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:10:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:10:14.180612 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:14.182667 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:14.196754 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57fead-be75-4d68-88c7-a5e828ebe16f service nova] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:14.201893 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2c393-a38d-47c8-9e40-b8665d73e7d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:14.202575 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c393-a38d-47c8-9e40-b8665d73e7d2 service placement] REQUEST_METHOD: `GET` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:14.202859 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c393-a38d-47c8-9e40-b8665d73e7d2 service placement] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:14.203164 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2c393-a38d-47c8-9e40-b8665d73e7d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:14.204903 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-14f2c393-a38d-47c8-9e40-b8665d73e7d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:14.216827 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2c393-a38d-47c8-9e40-b8665d73e7d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:14.218459 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2c393-a38d-47c8-9e40-b8665d73e7d2 service placement] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:14.230262 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1452/2892] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:10:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:10:17.743528 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:17.750163 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:17.812274 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467750d2-efad-4927-aec7-2f7538b7e9da service neutron] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:17.816934 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:17.817798 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:17.818107 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:17.818379 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] PATH_INFO: `/v3/users/0d16fe0fde164dd8b0652168fc438263` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:17.819597 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d16fe0fde164dd8b0652168fc438263)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:17.821684 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:18.002373 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d16fe0fde164dd8b0652168fc438263'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.079260 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d16fe0fde164dd8b0652168fc438263'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.079886 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d16fe0fde164dd8b0652168fc438263'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.087994 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d16fe0fde164dd8b0652168fc438263'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.101723 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d16fe0fde164dd8b0652168fc438263'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.106713 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1441/2893] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:10:17 2022] DELETE /identity/v3/users/0d16fe0fde164dd8b0652168fc438263 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:10:18.383508 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2c393-a38d-47c8-9e40-b8665d73e7d2 service placement] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:18.394201 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2c393-a38d-47c8-9e40-b8665d73e7d2 service placement] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:18.452108 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:18.453060 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:18.453349 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:18.453631 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] PATH_INFO: `/v3/projects/955b9d20401546f9a3f8477a9e67fc24` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:18.454910 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=955b9d20401546f9a3f8477a9e67fc24)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:18.456860 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:18.623546 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '955b9d20401546f9a3f8477a9e67fc24', 'request_id': 'req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.629270 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '955b9d20401546f9a3f8477a9e67fc24', 'request_id': 'req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.719252 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '955b9d20401546f9a3f8477a9e67fc24', 'request_id': 'req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.730930 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] The token cache is being invalidate because project 955b9d20401546f9a3f8477a9e67fc24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:10:18.735045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.735045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:18.735045 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1453/2894] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:10:18 2022] DELETE /identity/v3/projects/955b9d20401546f9a3f8477a9e67fc24 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:10:18.737480 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:18.738909 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:18.752884 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99b731-3041-4df3-9020-6e5608c5fe82 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:18.811301 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:18.812375 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:18.812772 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:18.813160 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] PATH_INFO: `/v3/users/15507a71243d4055a6f01ae36e5743e4` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:18.814612 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15507a71243d4055a6f01ae36e5743e4)` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:18.816782 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:18.823007 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:10:19.055932 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15507a71243d4055a6f01ae36e5743e4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.142069 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15507a71243d4055a6f01ae36e5743e4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.142701 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15507a71243d4055a6f01ae36e5743e4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.150584 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15507a71243d4055a6f01ae36e5743e4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.151317 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.notifications [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15507a71243d4055a6f01ae36e5743e4'} {{(pid=92133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.169344 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1442/2895] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 14 16:10:18 2022] DELETE /identity/v3/users/15507a71243d4055a6f01ae36e5743e4 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:10:19.412898 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] Authenticating user token {{(pid=92134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:10:19.415375 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.fernet_utils [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:10:19.428709 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913764b5-3f73-4eb1-b2ae-791bb3e5d8d8 admin admin] Validating token access rules against request {{(pid=92134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:10:19.487339 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2ffbc8434d34115805dcbc2a72dc671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd8c317cfa3fe4e0aa6bdd74dbc68a586', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:10:19.488212 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:10:19.488500 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] SCRIPT_NAME: `/identity` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:10:19.488776 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] PATH_INFO: `/v3/projects/454b2afb06404b2db5e42cceb572e3ac` {{(pid=92134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:10:19.490017 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] RBAC: Authorizing `identity:delete_project(project_id=454b2afb06404b2db5e42cceb572e3ac)` {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:10:19.492492 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] RBAC: Authorization granted {{(pid=92134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:10:19.513579 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:10:19.606331 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '454b2afb06404b2db5e42cceb572e3ac', 'request_id': 'req-66cd3b6c-4a99-45e4-957b-5be02d95203d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.612173 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '454b2afb06404b2db5e42cceb572e3ac', 'request_id': 'req-66cd3b6c-4a99-45e4-957b-5be02d95203d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.718596 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '454b2afb06404b2db5e42cceb572e3ac', 'request_id': 'req-66cd3b6c-4a99-45e4-957b-5be02d95203d'} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.731463 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] The token cache is being invalidate because project 454b2afb06404b2db5e42cceb572e3ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 16:10:19.732424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.732424 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG keystone.notifications [None req-66cd3b6c-4a99-45e4-957b-5be02d95203d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 16:10:19.734069 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: [pid: 92134|app: 0|req: 1454/2896] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 14 16:10:19 2022] DELETE /identity/v3/projects/454b2afb06404b2db5e42cceb572e3ac => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 16:10:29.165683 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:10:29.728044 dv-cinder-837164-3-iscsi devstack@keystone.service[92134]: DEBUG dbcounter [-] [92134] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 16:14:15.987654 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] Authenticating user token {{(pid=92133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 16:14:15.989177 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:16.130156 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30405c0-d1d8-4f45-a526-67c7fca464f7 admin admin] Validating token access rules against request {{(pid=92133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 16:14:16.189289 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf35ae9-6d52-403b-a470-6737f299aac8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97524081e59f4775b83d82edf737810e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3056a2f260314184a08f3a9101a335d2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 16:14:16.189997 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf35ae9-6d52-403b-a470-6737f299aac8 service placement] REQUEST_METHOD: `GET` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 16:14:16.190286 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf35ae9-6d52-403b-a470-6737f299aac8 service placement] SCRIPT_NAME: `/identity` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 16:14:16.190566 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf35ae9-6d52-403b-a470-6737f299aac8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 16:14:16.192393 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.fernet_utils [None req-5cf35ae9-6d52-403b-a470-6737f299aac8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 16:14:16.204110 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf35ae9-6d52-403b-a470-6737f299aac8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 16:14:16.205698 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf35ae9-6d52-403b-a470-6737f299aac8 service placement] RBAC: Authorization granted {{(pid=92133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 16:14:16.215550 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: [pid: 92133|app: 0|req: 1443/2897] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 14 16:14:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 16:14:26.211346 dv-cinder-837164-3-iscsi devstack@keystone.service[92133]: DEBUG dbcounter [-] [92133] Writing DB stats keystone:SELECT=89 {{(pid=92133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}